The way to Construct a Calorie Advisor App Utilizing GenAI?

Introduction

Synthetic Intelligence has many use instances, and among the greatest ones are within the Well being Business. It may possibly actually assist folks preserve a more healthy life. With the rising increase in generative AI, sure functions are made lately with much less complexity. One very helpful utility that may be constructed is the Calorie Advisor App. On this article, we’ll solely have a look at this, impressed by caring for our well being. We might be constructing a easy Calorie Advisor App the place we will enter the pictures of the meals, and the app will assist us calculate the energy of every merchandise current within the meals. This undertaking is part of NutriGen, specializing in well being by way of AI.

Studying Goal

• The App we might be creating on this article might be primarily based on primary Immediate engineering and picture processing methods.
• We might be utilizing Google Gemini Professional Imaginative and prescient API for our use case.
• Then, we’ll create the code’s construction, the place we’ll carry out Picture Processing and Immediate Engineering. Lastly, we’ll work on the Consumer Interface utilizing Streamlit.
• After that, we’ll deploy our app to the Hugging Face Platform for Free.
• We can even see among the issues we’ll face within the output the place Gemini fails to depict a meals merchandise and offers the flawed calorie rely for that meals.  We can even focus on totally different options for this downside.

Pre-Requisites

Let’s begin with implementing our undertaking, however earlier than that, please guarantee you could have a primary understanding of generative AI and LLMs. It’s okay if you recognize little or no as a result of, on this article, we might be implementing issues from scratch.

For Important Python Immediate Engineering, a primary understanding of Generative AI and familiarity with Google Gemini is required. Moreover, primary information of Streamlit, Github, and Hugging Face libraries is critical. Familiarity with libraries similar to PIL for picture preprocessing functions can also be helpful.

This text was revealed as part of the Knowledge Science Blogathon.

Mission Pipeline

On this article, we might be engaged on constructing an AI assistant who assists nutritionists and people in making knowledgeable choices about their meals selections and sustaining a wholesome way of life.

The circulate might be like this: enter picture -> picture processing -> immediate engineering -> remaining perform calling to get the output of the enter picture of the meals. This can be a transient overview of how we’ll strategy this downside assertion.

Overview of Gemini Professional Imaginative and prescient

Gemini Professional is a multimodal LLM constructed by Google. It was educated to be multimodal from the bottom up. It may possibly carry out effectively on varied duties, together with picture captioning, classification, summarisation, question-answering, and so forth. One of many fascinating information about it’s that it makes use of our well-known Transformer Decoder Structure. It was educated on a number of sorts of knowledge, lowering the complexity of fixing multimodal inputs and offering high quality outputs.

Step1: Creating the Digital Surroundings

Making a digital atmosphere is an effective observe to isolate our undertaking and its dependencies such that they don’t coincide with others, and we will all the time have totally different variations of libraries we’d like in several digital environments. So, we’ll create a digital atmosphere for the undertaking now. To do that, comply with the talked about steps beneath:

• Create an Empty folder on the desktop for the undertaking.
• Open this folder in VS Code.
• Open the terminal.

Write the next command:

pip set up virtualenv
python -m venv genai_project

You should use the next command if you happen to’re getting sa et execution coverage error:

Set-ExecutionPolicy RemoteSigned -Scope Course of

Now we have to activate our digital atmosphere, for that use the next command:

.genai_projectScriptsactivate

We’ve efficiently created our digital atmosphere.

Step Create Digital Surroundings in Google Colab

We are able to additionally create our Digital Surroundings in Google Colab; right here’s the step-by-step process to try this:

• Create a New Colab Pocket book
• Use the beneath instructions step-by-step

!which python
!python --version
#to test if python is put in or not

%env PYTHONPATH=
# setting python path atmosphere variable in empty worth guaranteeing that python
# will not seek for modules and packages in further listing. It helps
# in avoiding conflicts or unintended module loading.

!pip set up virtualenv 

# create digital atmosphere 
!virtualenv genai_project

!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

#It will assist obtain the miniconda installer script which is used to create
# and handle digital environments in python

!chmod +x Miniconda3-latest-Linux-x86_64.sh
# this command is making our mini conda installer script executable inside
# the colab atmosphere. 

!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/native
# that is used to run miniconda installer script and 
# specify the trail the place miniconda must be put in

!conda set up -q -y --prefix /usr/native python=3.8 ujson
#this may assist set up ujson and python 3.8 set up in our venv.

import sys
sys.path.append('/usr/native/lib/python3.8/site-packages/')
#it would enable python to find and import modules from a venv listing

import os
os.environ['CONDA_PREFIX'] = '/usr/native/envs/myenv'

# used to activate miniconda enviornment 

!python --version
#checks the model of python throughout the activated miniconda atmosphere

Therefore, we additionally created our digital atmosphere in Google Colab. Now, let’s test and see how we will make a primary .py file there.

!supply myenv/bin/activate
#activating the digital atmosphere

!echo "print('Hey, world!')" >> my_script.py
# writing code utilizing echo and saving this code in my_script.py file

!python my_script.py
#working my_script.py file

It will print Hey World for us within the output. So, that’s it. That was all about working with Digital Environments in Google Colab. Now, let’s proceed with the undertaking.

Step2: Importing Crucial Libraries

import streamlit as st
import google.generativeaias genai 
import os 
from dotenv import load_dotenv
load_dotenv()
from PIL import Picture

If you’re having hassle importing any of the above libraries, you possibly can all the time use the command “pip set up library_name” to put in it.

We’re utilizing the Streamlit library to create the essential consumer interface. The consumer will be capable of add a picture and get the outputs primarily based on that picture.

We use Google Generative to get the LLM and analyze the picture to get the calorie rely item-wise in our meals.

Picture is getting used to carry out some primary picture preprocessing.

Step3: Establishing the API Key

Create a brand new .env file in the identical listing and retailer your API key. You may get the Google Gemini API key from Google MakerSuite.

Step4: Response Generator Perform

Right here, we’ll create a response generator perform. Let’s break it down step-by-step:

Firstly, we used genes. Configure to configure the API we created from the Google MakerSuite Web site. Then, we made the perform get_gemini_response, which takes in 2 enter parameters: the enter immediate and the picture. That is the first perform that can return the output in textual content.

genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

def get_gemini_response(input_prompt, picture):

    mannequin = genai.GenerativeModel('gemini-pro-vision')

    response = mannequin.generate_content([input_prompt, image[0]])

    return response

Right here, we’re utilizing the ‘Gemini-pro-vision’ mannequin as a result of it’s multimodal. After calling our mannequin from the genie.GenerativeModel dependency, we’re simply passing in our immediate and the picture information to the mannequin. Lastly, primarily based on the directions supplied within the immediate and the picture information we fed, the mannequin will return the output within the type of textual content that represents the calorie rely of various meals gadgets current within the picture.

Step5: Picture Preprocessing

This perform checks if the uploaded_file parameter is None, which means the consumer has uploaded a file. If a file has been uploaded, the code proceeds to learn the file content material into bytes utilizing the getvalue() technique of the uploaded_file object. It will return the uploaded file’s uncooked bytes.

The bytes information obtained from the uploaded file is saved in a dictionary format underneath the key-value pair “mime_type” and “information.” The “mime_type” key shops the uploaded file’s MIME sort, which signifies the kind of content material (e.g., picture/jpeg, picture/png). The “information” key shops the uploaded file’s uncooked bytes.

The picture information is then saved in a listing named image_parts, which incorporates a dictionary with the uploaded file’s MIME sort and information.

def input_image_setup(uploaded_file):
    if uploaded_file isnotNone:
        #Learn the file into bytes
        bytes_data = uploaded_file.getvalue()
        image_parts = [
            {
                "mime_type":uploaded_file.type, 
                "data":bytes_data
            }
        ]
        return image_parts
    else:
        elevate FileNotFoundError("No file uploaded")

Step6: Creating the UI

So, lastly, it’s time to create the consumer interface for our undertaking. As talked about earlier than, we might be utilizing the Streamlit library to put in writing the code for the entrance finish.

## initialising the streamlit app
st.set_page_config(page_title="Energy Advisor App")
st.header("Energy Advisor App")
uploaded_file = st.file_uploader("Select a picture...", sort=["jpg", "jpeg", "png"])
picture = ""
if uploaded_file isnotNone:
    picture = Picture.open(uploaded_file)
    st.picture(picture, caption="Uploaded Picture", use_column_width=True)
submit = st.button("Inform me in regards to the complete energy")

Initially, we arrange the web page configuration utilizing set_page_config and gave the app a title. Then, we created a header and added a file uploader field the place customers can add photographs. St. Picture exhibits the picture that the consumer uploaded to the UI. Ultimately, there’s a submit button, after which we’ll get the outputs from our massive language mannequin, Gemini Professional Imaginative and prescient.

Step7: Writing the System Immediate

Now’s the time to be inventive. Right here, we’ll create our enter immediate, asking the mannequin to behave as an skilled nutritionist. It’s not crucial to make use of the immediate beneath; you may as well present your customized immediate. We’re asking our mannequin to behave a sure means for now. Primarily based on the enter picture of the meals supplied, we’re asking our mannequin to learn that picture information and generate the output, which is able to give us the calorie rely of the meals gadgets current within the picture and supply a judgment of whether or not the meals is wholesome or unhealthy. If the meals is dangerous, we ask it to provide extra nutritious alternate options to the meals gadgets in our picture. You possibly can customise it extra in line with your wants and get a superb method to preserve monitor of your well being.

Generally it won’t in a position to learn the picture information correctly, we’ll focus on options relating to this additionally on the finish of this text.

input_prompt = """

You're an skilled nutritionist the place it's essential see the meals gadgets from the 
picture and calculate the whole energy, additionally give the small print of all 
the meals gadgets with their respective calorie rely within the beneath fomat.

        1. Merchandise 1 - no of energy

        2. Merchandise 2 - no of energy

        ----

        ----

Lastly you may as well point out whether or not the meals is wholesome or not and in addition point out 
the proportion break up ratio of carbohydrates, fat, fibers, sugar, protein and 
different essential issues required in our food regimen. When you discover that meals just isn't wholesome 
then you need to present some different wholesome meals gadgets that consumer can have 
in food regimen.

"""
if submit:

    image_data = input_image_setup(uploaded_file)

    response = get_gemini_response(input_prompt, image_data)

    st.header("The Response is: ")

    st.write(response)

Lastly, we’re checking that if the consumer clicks the Submit button, we’ll get the picture information from the

input_image_setup perform we created earlier. Then, we go our enter immediate and this picture information to the get_gemini_response perform we created earlier.  We name all of the capabilities we created earlier to get the ultimate output saved in response.

Step8: Deploying the App on Hugging Face

Now’s the time for deployment. Let’s start.

Will clarify the best method to deploy this app that we created. There are two choices that we will look into if we need to deploy our app: one is Streamlit Share, and the opposite one is Hugging Face. Right here, we’ll use Hugging Face for the deployment; you possibly can attempt exploring deployment on Streamlit Share iFaceu if you’d like. Right here’s the reference hyperlink for that – Deployment on Streamlit Share

First, let’s shortly create the necessities.txt file we’d like for the deployment.

Open the terminal and run the beneath command to create a necessities.txt file.

pip freeze > necessities.txt1plainText

It will create a brand new textual content file named necessities. All of the undertaking dependencies might be out there there. If this causes an error, it’s okay. You possibly can all the time create a brand new textual content file in your working listing and replica and paste the necessities.txt file from the GitHub hyperlink I’ll present subsequent.

Now, just be sure you have these information helpful (as a result of that’s what we’d like for the deployment):

• app.py
• .env (for the API credentials)
• necessities.txt

When you don’t have one, take all these information and create an account on the cuddling face. Then, create a brand new house and add the information there. That’s all. Your app might be routinely deployed this fashion. Additionally, you will be capable of see how the deployment is happening in real-time. If some error happens, you possibly can all the time determine it out with the easy interface and, after all, the cuddling face group, which has plenty of content material on resolving some frequent bugs throughout deployment.

After a while, it is possible for you to to see the app working. Woo hoo! We’ve lastly created and deployed our calorie predictor app. Congratulations!!, You possibly can share the working hyperlink of the app with the family and friends you simply constructed.

Right here’s the working hyperlink to the app that we simply created – The Alorcalorieisor App

Let’s check our app by offering an enter picture to it:

Earlier than:

After:

Full Mission GitHub Hyperlink

Right here’s the entire github repository hyperlink that features supply code and different useful info relating to the undertaking.

You possibly can clone the repository and customise it in line with your necessities. Attempt to be extra inventive and clear in your immediate, as this may give your mannequin extra energy to generate right and correct outputs.

Scope of Enchancment

Issues that may happen within the outputs generated by the mannequin and their options:

Generally, there may very well be conditions the place you’ll not get the proper output from the mannequin. This will occur as a result of the mannequin was not in a position to predict the picture appropriately. For instance, if you happen to give enter photographs of your meals and your meals merchandise incorporates pickles, then our mannequin would possibly think about it one thing else. That is the first concern right here.

• One method to deal with that is by way of efficient immediate engineering methods, like few-shot immediate engineering, the place you possibly can feed the mannequin with examples, after which it would generate the outputs primarily based on the learnings from these examples and the immediate you supplied.
• One other resolution that may be thought-about right here is creating our customized information and fine-tuning it. We are able to create information containing a picture of the meals merchandise in a single column and an outline of the meals gadgets current within the different column. It will assist our mannequin study the underlying patterns and predict the gadgets appropriately within the picture supplied. Thus, getting extra right outputs of the calorie rely for the photographs of the meals is crucial.
• We are able to take it additional by asking the consumer about his/her vitamin objectives and asking the mannequin to generate outputs primarily based on that. (This manner, we can tailor the outputs generated by the mannequin and provides extra user-specific outputs.)

Conclusion

We’ve delved into the sensible utility of Generative AI in healthcare, specializing in the creation of the Calorie Advisor App. This undertaking showcases the potential of AI to help people in making knowledgeable choices about their meals selections and sustaining a wholesome way of life. From organising our surroundings to implementing picture processing and immediate engineering methods, we’ve coated the important steps. The app’s deployment on Hugging Face demonstrates its accessibility to a wider viewers. Challenges like picture recognition inaccuracies have been addressed with options similar to efficient immediate engineering. As we conclude, the Calorie Advisor App stands as a testomony to the transformative energy of Generative AI in selling well-being.

Key Takeaways

• We’ve mentioned rather a lot to this point, Beginning with the undertaking pipeline after which a primary introduction to the big language mannequin Gemini Professional Imaginative and prescient.
• Then, we began with the hands-on implementation. We created our digital atmosphere and API key from Google MakerSuite.
• Then, we carried out all our coding within the created digital atmosphere. Additional, we mentioned how you can deploy the app on a number of platforms, similar to Hugging Face and Streamlit Share.
• Aside from that, we thought-about the doable issues that may happen, and mentioned soluFaces to these issues.
•  Therefore, it was enjoyable engaged on this undertaking. Thanks for staying until the tip of this text; I hope you bought to study one thing new.

Continuously Requested Questions

The media proven on this article just isn’t owned by Analytics Vidhya and is used on the Writer’s discretion.