ML Guide to Train Food Recognition and Classification Model

Table of Contents

1. Introduction
2. Deep Learning in Food Image Classification
3. Implementing a Model
4. Application of Food Recognition and Classification
5. Conclusion
6. Frequently Asked Questions

Introduction

Food recognition and classification is a fascinating area of computer vision that focuses on teaching machines to recognize different types of food from images. With the rise of social media and the increasing interest in healthy eating, there's a growing demand for automated systems that can accurately identify and categorize food items.

Imagine pointing your phone camera at your lunch and having it instantly identify the dish, its ingredients, and even its estimated calorie count! This fascinating possibility is becoming increasingly real thanks to deep learning techniques.

By using advanced techniques like deep learning, researchers and developers are making significant progress in creating algorithms that can distinguish between various foods with impressive accuracy.

Deep learning in Food image classification

Deep learning has revolutionized food image classification. Convolutional Neural Networks (CNNs), a type of deep learning architecture, excel at recognizing patterns in images.

Unlike traditional methods that rely on handcrafted features, deep learning models, such as Convolutional Neural Networks (CNNs), can autonomously discover complex patterns and relationships within food images.

By training on large datasets of labeled food images, these models can accurately classify different types of food with high precision.

Techniques like transfer learning and data augmentation further enhance the performance and robustness of deep learning-based classifiers. As a result, deep learning has become the cornerstone of food image classification.

This technology has potential applications in dietary assessment, food safety, personalized recipe recommendations, and more, making it a key ingredient in the future of food technology.

Implementing a Model

In this blog, we will be creating our own food image classification using transfer learning. We will be using the MobilenetV2 model and kaggle dataset for building our model.

1. First, we will be importing the required libraries.

import numpy as np
import pandas as pd
from pathlib import Path
import os.path
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
import tensorflow as tf
from sklearn.metrics importclassification_report

2. Now let's read the dataset and create a single DataFrame that concatenates all the sampled data from each category.

image_dir = Path(r'C:\Users\KIIT\Desktop\labelstudio\food\images')
filepaths = list(image_dir.glob(r'**/*.jpg'))
labels = list(map(lambda x: os.path.split(os.path.split(x)[0])[1], filepaths))
filepaths = pd.Series(filepaths, name='Filepath').astype(str)
labels = pd.Series(labels, name='Label')
images = pd.concat([filepaths, labels], axis=1)
category_samples = []
for category in images['Label'].unique():
category_slice = images.query("Label == @category")
category_samples.append(category_slice.sample(100, random_state=1))
image_df = pd.concat(category_samples, axis=0).sample(
frac=1.0, random_state=1).reset_index(drop=True)

3. Let’s see the value count for all the labels.

image_df['Label'].value_counts()

4. Now we will divide our dataset into training and testing in the ratio of 70:30.

train_df, test_df = train_test_split(
image_df, train_size=0.7, shuffle=True, random_state=42)

5. Now we will set up two image data generators using TensorFlow's tf.keras.preprocessing.image.ImageDataGenerator class.

train_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input,
validation_split=0.2
)
test_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input
)

1) train_generator is configured for generating training data. It preprocesses images using the MobileNet V2 preprocessing function and allocates 20% of the data for validation (specified by validation_split=0.2).

2) test_generator is configured for generating test data. It also preprocesses images using the MobileNet V2 preprocessing function but doesn't split the data for validation, implying it's solely for testing purposes.

6. Now we will set up three image generators for training, validation, and testing data using the flow_from_dataframe method provided by TensorFlow's ImageDataGenerator class.

train_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=True,
seed=42,
subset='training'
)
val_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=True,
seed=42,
subset='validation'
)
test_images = test_generator.flow_from_dataframe(
dataframe=test_df,
x_col='Filepath',
y_col='Label',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=32,
shuffle=False
)

6. We will be using MobileNetV2 to train our model.

pretrained_model = tf.keras.applications.MobileNetV2(
input_shape=(224, 224, 3),
include_top=False,
weights='imagenet',
pooling='avg'
)
pretrained_model.trainable = False

7. Now we will add additional dense layers on top of the pretrained MobileNetV2 model's output and construct a new model suitable for the specific classification task.

inputs = pretrained_model.input
x = tf.keras.layers.Dense(128, activation='relu')(pretrained_model.output)
x = tf.keras.layers.Dense(128, activation='relu')(x)
outputs = tf.keras.layers.Dense(101, activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
print(model.summary())

8. Now let’s train the model. We will train our model for 50 epochs. We have also used EarlyStopping to stop training early if the validation loss doesn't improve for three consecutive epochs and restore the best weights.

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
history = model.fit(
    train_images,
    validation_data=val_images,
    epochs=50,
    callbacks=[
        tf.keras.callbacks.EarlyStopping(
            monitor='val_loss',
            patience=3,
            restore_best_weights=True)])

9. Now let's evaluate the model and print the test accuracy.

results = model.evaluate(test_images, verbose=0)
print("Test Accuracy: {:.2f}%".format(results[1] * 100))

We have achieved an accuracy of 82.74%.

Application of Food recognition and classification

Food recognition and classification using deep learning have various applications:

Dietary assessment

Imagine effortlessly tracking your daily calorie intake by simply taking pictures of your meals. Deep learning models can analyze food images and estimate their nutritional content, aiding individuals in managing their health and dietary goals.

Food safety and inspection

Automating food safety inspections can significantly improve efficiency and accuracy. Deep learning models can be trained to detect potential contaminants or irregularities in food items, ensuring consumer safety.

Personalized recipe recommendations

Deep learning algorithms can suggest suitable recipes, helping you explore new culinary horizons. Just take a picture of the ingredients you have on hand, and get a personalized recipe recommendation.

Restaurant menu analysis

Deep learning can empower users with real-time information about allergens and dietary restrictions present in menu items, promoting informed dining choices.

Conclusion

Deep learning has shown great promise in the field of food recognition and classification, enabling automated systems to identify and categorize different types of food accurately from images.

With continued research and advancements in deep learning techniques, we can expect further improvements in the performance and scalability of food recognition systems, making them valuable tools for dietary analysis, health monitoring, and food-related applications.

Looking for high quality training data to train your Food Recognition and Classification Model? Talk to our team to get a tool demo.

Frequently Asked Questions

Q1. What is deep learning in food image classification?

Deep learning in food image classification refers to the use of advanced machine learning techniques, specifically CNNs, to automatically analyze and categorize images of different types of food.

These deep learning models are trained on large datasets of labeled food images to learn and extract meaningful features that distinguish between various food items.

Q2. How does deep learning improve food image classification accuracy?

Deep learning models, such as CNNs, have the ability to automatically learn hierarchical representations of features from raw pixel data. Unlike traditional methods that rely on handcrafted features, deep learning models can discover complex patterns and relationships within food images, leading to higher accuracy in classification tasks.

Looking for high quality training data to train your food recognition and classification model ? Talk to our team to get a tool demo.