Legs or Hot Dogs image classifier

Since the beginning of time, people have yearned to know whether an image is legs, or hotdogs (just search Google Images to see more examples).

This age old problem was the plot for this week’s Silicon Valley episode Hot Dog or Not Hot Dog?

Machine learning makes it easy for us to retrain an image classifier. I threw together some training images in GitHub (in this legs-or-hotdogs-images repo) and retrained a classifier.

I only have 120 training images in total, so my expectations weren’t very high. However, it did do well on easy images. For the image above, it classified it as hot dogs with a 90% confidence.

How to

Create a folder in the home directory on your local machine:

mkdir $HOME/tf_files

Download and start my Docker container. We are using my container instead of the standard TensorFlow one because mine has helpful tools (like git) installed already (see my earlier posts if you need more context).

docker run -it -v $HOME/tf_files:/tf_files danjarvis/tensorflow-android:1.0.0

If you are new to Docker, this post explains more about Docker essentials.

In the Docker container (which should have a # prompt), clone the legs-or-hotdogs-images repo so you have access to the training images.

cd /tf_files
git clone --depth 1 --branch master https://github.com/daj/legs-or-hotdogs-images.git photos
rm -rf photos/.git

The last line here deletes the .git folder, otherwise that causes errors when trying to retrain our TensorFlow model.

We should now have a legs and a hot dogs folder.

cd /photos
ls -1

Now we can do the retraining:

cd /tensorflow
python tensorflow/examples/image_retraining/retrain.py \
--bottleneck_dir=/tf_files/bottlenecks \
--how_many_training_steps 500 \
--model_dir=/tf_files/inception \
--output_graph=/tf_files/retrained_graph.pb \
--output_labels=/tf_files/retrained_labels.txt \
--image_dir /tf_files/photos

By default Docker will discard your changes when you close the container, so do not close the command prompt window!

We aren’t planning to use this model in an Android app, so we can jump straight to testing some images by following these instructions from TensorFlow for Poets.

We just need to copy the label_files.py script (referenced in the codelabs above) from GitHub.

cd /tf_files
curl -O https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/image_retraining/label_image.py

Now we can feed in some previously unseen test images and see how well our classifier does, e.g.

# python label_image.py test_01_hotdogs.jpg
hot dogs (score = 0.89646)
legs (score = 0.10354)

The results

89% confidence hot dogs. Very good!

59% confidence legs. Wrong!

91% confidence legs. Very good!

50% confidence legs. Even though it was correct, I consider this a fail because the confidence is the same as a coin toss.

Next steps

For next steps, I’d like to feed in more training images and see if I can improve on the results above. If you are interested in helping, please make a pull request with more training images to the legs-or-hotdogs-images repo!

It was hard to find suitable images, especially ones in the public domain (only about half of ours are public domain licensed). Thanks to Stephen Tan, who helped collect the first 120 training images.

One of my earlier posts mentions the ImageNet training data. This might be a source of training images…

• http://image-net.org/search?q=hot+dog There are >1250 pictures of hot dogs…though oddly they seem to almost all be chili dogs. The photos also seem to mostly include buns and toppings, so they might not be so good for “legs or hot dogs” classification. :-/
• http://image-net.org/search?q=leg Probably doesn’t show enough flesh tone for our training needs. Some other synsets show more skin, but might not be close enough views of the legs, e.g. http://image-net.org/search?q=swimsuit and http://image-net.org/search?q=skirt

If you want to find out more about using this TensorFlow model on Android, see Using a pre-trained TensorFlow model on Android.

Footnote: the Silicon Valley episode aired on Sunday 14th May 2017. I had this idea months before — I created my legs-or-hotdogs-images repo on 26th Feb 2017. :-)