Back to home


Filter posts to Software, Quarantine or Machine-learning

For this year's final project in "Intro to Graduate ML", Daniel Jeong, Anush Devadhasan, and myself were tasked with the facial expression recognition problem (check out our in-depth presentation here: The objective is fairly simple: given a 48x48 greyscale image of a face, predict the emotion on that face. The amount of variation between faces and features our brains automatically account for makes this task deceptively difficult for machines. Originally, we used the FER2013 dataset but quickly discovered that it was mired with non-face images and incorrect labels (which seems to indicate that the dataset was gathered and labeled by a machine, ironically). But even if everything was properly curated and labeled, the problem definition is flawed: people aren't usually displaying just one emotion. Microsoft recognized this in 2016 and created the FER+ dataset, a probabilistic labeling of the original FER2013 dataset where 10 crowdsourced individuals gauged the expression in each image. The startling result is that people seemed to have different interpretations of emotion as well, because most images are split between multiple categories (with a large bias toward Neutral labels). The shortcomings of humans and machines for this task spurred us to find a more machine-interpretable mode of emotion learning. Specifically, we wanted to find a continuous, low-dimensional space that allowed us to represent and interpolate emotive facial expressions in a way that was machine-interpretable. Enter the discriminator-VAE, a VAE with a 2-pronged decoder, one which would decode the latent space for an image reconstruction and another which would attempt to pull the softmax FER+ prediction from the encoding. The results were conducive to clustering and allowed image reconstruction, which we used to project multiple different emotions onto a single face. However, while some clusters showed clear emotions, others showed prominent representations of face orientation, age, and sex. Overall this was a step above other methods for representing human emotion, but the small/unbalanced dataset prevented us from exploring further at the end of our semester.

Dec. 26, 2020

What do political Twitter accounts, private ML datasets, sold-out hype brands, and smart-fridge-based DDOS attacks (see:…) all have in common? They're mostly facilitated by bots. The fraction of human to bot web activity was about 0.5 in 2019 and climbing. Many bots do useful things like saving videos or automatically summarizing news articles. But good bots now make up a smaller portion of traffic than bad bots, and that's just the known ones. What was once a wacky and original place full of real people now feels like an echo-chamber of Cleverbots and advertisements. I'm not proposing a solution, just raising awareness that the human part of the internet is being poisoned with no end in sight.

Sept. 6, 2020

For a long time, pop-up ads were considered the worst part of the internet. Design-wise they're perfect: display your advertisements in a way that doesn't associate with whatever content is actually on a user-created page. However the sad truth is that they're wildly annoying, so much so that they have been almost completely phased out of existence. The pop-up ad is not unlike having a great dinner conversation and suddenly the waitstaff begins belting out happy-birthday. Luckily, it's been a long time since we've encountered either pop-ups or forced public birthday songs because popular annoyance has made itself painfully clear. Unfortunately, the pop-up has a successor: the mailto link. Similar to how LinkedIn found legitimacy, the mailto link is a pop-up masquerading as a professional courtesy. People seem to think that a potential employer or employee will enjoy having Outlook or Apple Mail pop up onto their desktop (only after the 8-10 second startup procedure). Even if you're the 1% of people who's configured their personal email app to open upon clicking a mailto, it completely ignores the only safety precaution you have when sending a first-impression email: adding the recipient address last to stop yourself from accidentally sending your half baked please-reopen-the-quiz appeal. In the name of being part of the solution and not part of the problem, please see the copy-to-clipboard javascript on this page linked to my email button!

Aug. 2, 2020

Everyone I've spoken with has a personal takeaway from their time in quarantine; here's mine: What we put online is now an essential part of how we view, interact with, and connect to each other. Not really a mind-blowing discovery, but for the first time ever our daily in-person interactions are completely shadowed by what we put online. Still, I think of most social medias as supplementary to in-person life but please tell me how to put my whole conscious on Twitter if you know. This site is a first-attempt at putting more of my work online. Especially as I'm starting a new life in Pittsburgh at a whole new university, 144 characters felt limited. The site isn't impressive tech-wise (django-gunicorn-nginx stack on AWS Lightsail) but what I love about this site is that I made it. I thought through every part of this site, and when you're here you're using something I made. The interaction feels a little more genuine to me that way. Also the little main-page icons light up when you mouse over them and that's sick."