New AI model suggests recipes based on available food ingredients

The training of the AI model

The computational model was trained using over 2,000 images of open refrigerators containing various raw food items. As a result, the AI model was able to create a 100-page recipe book consisting of various unique recipes using these image inputs. 

“For the first time, an AI chef or cook seems not only possible but offers some enhanced capabilities to augment human recipe libraries in pragmatic ways. The work generates a 100-page recipe book featuring the thirty top ingredients using over 2000 refrigerator images as initializing lists.” the paper noted.

The AI method is based on models that can recognize objects in images along with a text generator model to give out recipes. Specifically, they used application programming interfaces (APIs) and GPT-4, an OpenAI large language model (LLM), to generate recipes based on text or images provided by the users.

The researchers hope to incorporate this AI model into a smartphone app or other software tools to generate new and innovative recipes. 

Study abstract:

The AI community has embraced multi-sensory or multi-modal approaches to advance this generation of AI models to resemble expected intelligent understanding. Combining language and imagery represents a familiar method for specific tasks like image captioning or generation from descriptions. This paper compares these monolithic approaches to a lightweight and specialized method based on employing image models to label objects, then serially submitting this resulting object list to a large language model (LLM). This use of multiple Application Programming Interfaces (APIs) enables better than 95% mean average precision for correct object lists, which serve as input to the latest Open AI text generator (GPT-4). To demonstrate the API as a modular alternative, we solve the problem of a user taking a picture of ingredients available in a refrigerator, then generating novel recipe cards tailored to complex constraints on cost, preparation time, dietary restrictions, portion sizes, and multiple meal plans. The research concludes that monolithic multimodal models currently lack the coherent memory to maintain context and format for this task and that until recently, the language models like GPT-2/3 struggled to format similar problems without degenerating into repetitive or non-sensical combinations of ingredients. For the first time, an AI chef or cook seems not only possible but offers some enhanced capabilities to augment human recipe libraries in pragmatic ways. The work generates a 100-page recipe book featuring the thirty top ingredients using over 2000 refrigerator images as initializing lists.