Hi! 2023 was my fourth year living in Tokyo and working at Mercari. Here’s how I spent the year.

Highlights

1. Completed a manual transmission driving course at a driving school and got my driving license in Japan
2. Went on a 7-day long road trip in Hokkaido and a 5-day one in Kyushu
3. Tried skydiving for the first time as a birthday present for myself at Tokyo Skydiving Club in Saitama
4. Gave a 30-minute tech talk at PyCon APAC 2023 in Tokyo
5. Attended EMNLP 2023 in Singapore
6. Cycled the 80km long Shimanami-kaido route in Shikoku (in a single day)
7. Saw Megadeth and Dream Theater live
8. Started taking weekly piano lessons
9. Went on a family trip with my parents to Amritsar
10. Moved apartment from Asakusa to Shibuya

Work

Towards the end of 2022, I was working on correctly routing customer inquiries to suitable CS agents. To route correctly, we need to identify the type of problem the customer is facing based on the inquiry. After that, we can route the inquiry to the suitable agent since we already know the types of problems each CS agent can handle. Our goal was to use machine learning to identify the type of problem for each inquiry to increase accuracy. I had already finished a PoC for this by the end of 2022.

I spent the first quarter of 2023 designing & defining suitable metrics (both ML and business) for this project, productionizing the ML model, and running and analyzing the A/B test. We had a successful A/B test and 100% release to production. The ML model increased accuracy by 8.2% and reduced the average number of transfers per reply by 26.4%. I gave a talk about this project at PyCon APAC 2023.

I spent the latter part of the year developing an automatic reply system for customer inquiries using GPT models. Our goal was to send dynamically generated responses using GPT models and see if it had positive effects on our business metrics. I started by analyzing the raw data to identify categories where sending completely generated replies without human intervention would be suitable. I spent lots of time tuning prompts and adding suitable contexts for each of those categories. We did manual and human evaluations and iterated multiple times until the response quality passed our thresholds. I led the design, implementation, and A/B test of this system. After running the test, we found that the impact on our goal metrics was positive but not high enough to justify using the current version of the system in production. We went back to the drawing board to improve the solution.

Besides these two major projects, I was also involved in the hiring process for ML engineers and interns. Maybe it was the job market or maybe we had more interviewers for ML hiring, but compared to 2022, I had to do fewer evaluations. I evaluated 6 take-home assignments and conducted 3 technical interviews.

As for fun things, during a hackathon I built a Chrome extension mostly for personal use. It has prompt presets for my most frequent tasks for which I consult LLMs, and I can quickly run a query without having to switch to a new or different tab in the browser. The demo below shows one of my use cases.

Conferences

I gave my first-ever tech talk at a conference at PyCon APAC 2023 in Tokyo, Japan. It was a 30-minute talk about how we use machine learning at Mercari for efficiently routing customer inquiries to suitable CS agents. The talk video is here and more details are here.

I attended the EMNLP (Conference on Empirical Methods in Natural Language Processing) in Singapore. I organized a craft beer meetup for the conference attendees and met some fun people. I got to see the latest work people in academia are doing, and as expected, there was a lot of stuff about LLMs. I really loved the keynote at the end of the conference by Christopher Manning. I also had a lot of fun trying night photography around Singapore after attending the conference during the day. I really love these two below. Check this album on Flickr for more.

Driving

I graduated from a driving school and got my driving license. Getting a driving license is a long process in Japan if you’re doing it from scratch. It took me about 5 months from starting the classes at the driving school to getting the license. After doing a couple of small driving trips near Tokyo, I went on a 7-day, 1800-kilometer road trip in Hokkaido and a 5-day, 1200-kilometer one in Kyushu.

Moving Apartment

Moving apartments in Tokyo/Japan takes a lot of effort and paperwork and is mentally and physically exhausting. After living in Asakusa for 4 years, I moved to Shibuya-ku, and now I’m staying pretty close to Shinjuku as well. I created this checklist of tasks which you can use when you’re moving.

That wraps up the update for 2023. I’m looking forward to where 2024 takes me!