MGIE can crop, resize, flip, and add filters to images via text prompts.
The MGIE model takes advantage of a large multimodal language model to interpret user commands and perform pixel-level processing.
Apple worked with the University of California, Santa Barbara to develop the MGIE model, which can perform simple to complex image editing tasks, such as changing an object in an image to give it a different look or increase its brightness.
The model combines two different implementations of a large multimodal language model: it learns to interpret user input and then visualizes what the changes look like.
When users use MGIE templates to edit images, they need to write what they want to change in the image.
The paper demonstrates the effectiveness of MGIE in improving automatic metrics and human evaluation while maintaining the effectiveness of competitive inference.
“The MGIE model infers clear, visually aware intentions rather than brief, vague instructions, leading to judicious image editing,” Apple researchers said in their paper.
Apple researchers are conducting extensive research on all aspects of deployment to show that the MGIE model effectively improves performance while maintaining competitiveness.
Apple has made the MGIE model available for download via GitHub, where users can find code, data, and pre-trained models.
The company offers a demo showing how to use MGIE to perform various editing tasks. Users can also experience MGIE via a web demo hosted at Hugging Face Spaces, a platform for sharing and collaborating on machine learning projects.
Some image generation AI models, such as DALL-E 3 from OpenAI, can perform simple editing tasks on images generated from text input.
Adobe has an AI-based image editing model and the Firefly AI model supports features like generative fill for adding generated backgrounds to images.
Unlike Microsoft, Meta and Google, Apple is not known for producing artificial intelligence, although CEO Tim Cook has said that Apple hopes to add more AI features to its devices this year.
In December, Apple researchers released an open source machine learning framework called MLX to make it easier to train AI models on Apple Silicon chips.