ML Guide to Train Food Recognition and Classification Model

Table of Contents

1. Introduction
2. Deep Learning in Food Image Classification
3. Implementing a Model
4. Application of Food Recognition and Classification
5. Conclusion
6. Frequently Asked Questions

Introduction

Think about the last time you looked at a plate of food and wondered, What’s in this dish? How healthy is it? 

Now imagine if technology could answer those questions for you—instantly and effortlessly. 

Food recognition is no longer just a futuristic dream; it's rapidly becoming a reality, driven by advances in AI and deep learning.

But here's the thing: while we’re surrounded by food apps and calorie calculators, most still rely on manual input, leaving room for guesswork and errors. In a world where food habits are deeply tied to health and lifestyle, this gap is a missed opportunity. 

The solution? 

Teaching machines to not just see food but understand it—breaking down dishes into meaningful insights.

This post is not just about building a model but about empowering you to leverage the latest in AI for smarter food recognition. 

Using CNNs, transfer learning, and real-world datasets, you can create a system capable of distinguishing between a cappuccino and a macchiato, or even identifying hidden ingredients in complex meals. 

Curious to see how technology can elevate how we interact with food? Keep reading to find out!

Deep learning in Food image classification

Deep learning has revolutionized food image classification. Convolutional Neural Networks (CNNs), a type of deep learning architecture, excel at recognizing patterns in images.

Unlike traditional methods that rely on handcrafted features, deep learning models, such as Convolutional Neural Networks (CNNs), can autonomously discover complex patterns and relationships within food images.

By training on large datasets of labeled food images, these models can accurately classify different types of food with high precision.

Techniques like transfer learning and data augmentation further enhance the performance and robustness of deep learning-based classifiers. As a result, deep learning has become the cornerstone of food image classification.

This technology has potential applications in dietary assessment, food safety, personalized recipe recommendations, and more, making it a key ingredient in the future of food technology.

Implementing a Model

In this blog, we will be creating our own food image classification using transfer learning. We will be using the MobilenetV2 model and kaggle dataset for building our model.

1. First, we will be importing the required libraries.

import numpy as np
import pandas as pd
from pathlib import Path
import os.path
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
import tensorflow as tf
from sklearn.metrics importclassification_report

2. Now let's read the dataset and create a single DataFrame that concatenates all the sampled data from each category.

image_dir = Path(r'C:\Users\KIIT\Desktop\labelstudio\food\images')
filepaths = list(image_dir.glob(r'**/*.jpg'))
labels = list(map(lambda x: os.path.split(os.path.split(x)[0])[1], filepaths))
filepaths = pd.Series(filepaths, name='Filepath').astype(str)
labels = pd.Series(labels, name='Label')
images = pd.concat([filepaths, labels], axis=1)
category_samples = []
for category in images['Label'].unique():
category_slice = images.query("Label == @category")
category_samples.append(category_slice.sample(100, random_state=1))
image_df = pd.concat(category_samples, axis=0).sample(
frac=1.0, random_state=1).reset_index(drop=True)

3. Let’s see the value count for all the labels.

image_df['Label'].value_counts()

4. Now we will divide our dataset into training and testing in the ratio of 70:30.

train_df, test_df = train_test_split(
image_df, train_size=0.7, shuffle=True, random_state=42)

5. Now we will set up two image data generators using TensorFlow's tf.keras.preprocessing.image.ImageDataGenerator class.

train_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input,
validation_split=0.2
)
test_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input
)

1) train_generator is configured for generating training data. It preprocesses images using the MobileNet V2 preprocessing function and allocates 20% of the data for validation (specified by validation_split=0.2).

2) test_generator is configured for generating test data. It also preprocesses images using the MobileNet V2 preprocessing function but doesn't split the data for validation, implying it's solely for testing purposes.

6. Now we will set up three image generators for training, validation, and testing data using the flow_from_dataframe method provided by TensorFlow's ImageDataGenerator class.

train_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=True,
seed=42,
subset='training'
)
val_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=True,
seed=42,
subset='validation'
)
test_images = test_generator.flow_from_dataframe(
dataframe=test_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=False
)

6. We will be using MobileNetV2 to train our model.

pretrained_model = tf.keras.applications.MobileNetV2(
input_shape=(224, 224, 3),
include_top=False,
weights='imagenet',
pooling='avg'
)
pretrained_model.trainable = False

7. Now we will add additional dense layers on top of the pretrained MobileNetV2 model's output and construct a new model suitable for the specific classification task.

inputs = pretrained_model.input
x = tf.keras.layers.Dense(128, activation='relu')(pretrained_model.output)
x = tf.keras.layers.Dense(128, activation='relu')(x)
outputs = tf.keras.layers.Dense(101, activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
print(model.summary())

8. Now let’s train the model. We will train our model for 50 epochs. We have also used EarlyStopping to stop training early if the validation loss doesn't improve for three consecutive epochs and restore the best weights.

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
history = model.fit(
    train_images,
    validation_data=val_images,
    epochs=50,
    callbacks=[
        tf.keras.callbacks.EarlyStopping(
            monitor='val_loss',
            patience=3,
            restore_best_weights=True)])

9. Now let's evaluate the model and print the test accuracy.

results = model.evaluate(test_images, verbose=0)
print("Test Accuracy: {:.2f}%".format(results[1] * 100))

We have achieved an accuracy of 82.74%.

Application of Food recognition and classification

Food recognition and classification using deep learning have various applications:

Dietary assessment

Imagine effortlessly tracking your daily calorie intake by simply taking pictures of your meals. Deep learning models can analyze food images and estimate their nutritional content, aiding individuals in managing their health and dietary goals.

Food safety and inspection

Automating food safety inspections can significantly improve efficiency and accuracy. Deep learning models can be trained to detect potential contaminants or irregularities in food items, ensuring consumer safety.

Personalized recipe recommendations

Deep learning algorithms can suggest suitable recipes, helping you explore new culinary horizons. Just take a picture of the ingredients you have on hand, and get a personalized recipe recommendation.

Restaurant menu analysis

Deep learning can empower users with real-time information about allergens and dietary restrictions present in menu items, promoting informed dining choices.

Conclusion

Deep learning has shown great promise in the field of food recognition and classification, enabling automated systems to identify and categorize different types of food accurately from images.

With continued research and advancements in deep learning techniques, we can expect further improvements in the performance and scalability of food recognition systems, making them valuable tools for dietary analysis, health monitoring, and food-related applications.

Looking for high quality training data to train your Food Recognition and Classification Model? Talk to our team to get a tool demo.

Frequently Asked Questions

Q1. What is deep learning in food image classification?

Deep learning in food image classification refers to the use of advanced machine learning techniques, specifically CNNs, to automatically analyze and categorize images of different types of food.

These deep learning models are trained on large datasets of labeled food images to learn and extract meaningful features that distinguish between various food items.

Q2. How does deep learning improve food image classification accuracy?

Deep learning models, such as CNNs, have the ability to automatically learn hierarchical representations of features from raw pixel data. Unlike traditional methods that rely on handcrafted features, deep learning models can discover complex patterns and relationships within food images, leading to higher accuracy in classification tasks.

Looking for high quality training data to train your food recognition and classification model ? Talk to our team to get a tool demo.