MENLO PARK — Imagine snapping a picture of a mouthwatering dish and instantly receiving a recipe to recreate it at home. This futuristic concept is now a reality, thanks to an innovative project by Facebook AI researchers. The project, called “inverse cooking,” is designed to analyze photos of food and generate recipes, complete with a list of ingredients and step-by-step instructions.

While still in the research phase, this AI-driven system has the potential to revolutionize how we approach cooking. Unlike previous methods that relied on matching images to a database of existing recipes, this new system uses advanced computer vision to identify ingredients directly from the photo and then predict a recipe. The system can’t yet determine exact details like the type of flour or the precise cooking temperature, but it creates a recipe that closely resembles the dish in the image.

“Our system goes beyond simple image retrieval by splitting the process into two stages: identifying ingredients and generating a recipe,” explains Michal Drozdzal, a research scientist at Facebook AI. “This approach allows us to outperform traditional methods and generate more accurate and plausible recipes.”

The AI leverages two neural networks to achieve this: one for recognizing ingredients and another for creating the recipe. The system has been trained on the Recipe1M dataset, which includes nearly 17,000 ingredients and around 1 million recipes. Through this training, the AI learns to predict which ingredients typically appear together and how they are used in cooking.

The success of this project is particularly impressive given the complexity of food recognition, a notoriously difficult challenge in AI. Food items can vary significantly in appearance based on preparation methods and cooking stages, adding layers of complexity to the task. However, the AI’s ability to make educated guesses about ingredients and cooking methods from images alone is a significant step forward in the field.

Drozdzal and his colleague, Adriana Romero, who co-led the project, have demonstrated the system’s capabilities with various dishes, including complex ones like paella. The AI’s recipe predictions have been validated by human testers, who found the AI-generated recipes to be more successful than those produced by traditional retrieval systems.

Beyond its culinary applications, the principles behind inverse cooking could be applied to other areas, such as analyzing images of people to infer activities or cultural events. The project is still evolving, with ongoing work to refine the AI’s ability to predict ingredients and handle visually similar foods.

“We’re excited about the possibilities this technology offers,” says Romero. “While we haven’t yet tested the recipes ourselves in the kitchen, the system’s ability to generate detailed and plausible recipes from a simple photo is a promising advancement in AI research.”

This breakthrough in AI-generated recipes not only showcases the potential of machine learning in everyday life but also hints at a future where technology can seamlessly assist in creative tasks, making cooking and other activities more accessible and enjoyable for everyone.