Image generating models have spread from the process of extracting images from the prompt at once to the process of repeatedly improving the generating proposal using search, code execution, and self-correction. Additional inference time calculations are used not only to increase the number of candidates, but also to re-select the method of generating.
Return the search results and code drawings to the generated conditions.
Developed by Meta Superintelligence Labs, the research department of Meta, the image generation model Muse Image not only responds directly to the prompt to the image, but also acts as an agent that calls for search and code execution tools.
The search tool retrieves real facts, up-to-date information, and visual references from the web. With Meta’s internal comparisons, the use of search improved factuality for knowledge-intensive instructions dealing with current events and real facts.
The coding tool generates diagrams and QR codes and transmits the rendered diagrams to the model by adding numbers and arrangements to the result of the drawing assembled in the code, generating images in addition to instructions of words only.
Choose a new way of generating according to the error range.
If there is a minor problem with the initial generator, Muse Image will edit the part locally.If the range of modifications is large, it will be re-generated from scratch and switch to tool use if you need factual material.
This self-adjustment was not designed as an individual action in advance.Meta says that enhanced learning, which is rewarded as much as good images, resulted in a behavior that inspects and improves generating ideas.
Additional calculations instead of candidate generations.
By increasing inference calculations, Muse Image deepened inference and increased the number of tool calls and self-corrections.In Meta’s internal comparison, the eloscore based on human preferences improved approximately per line compared to the total amount of calculations used for text inference and image generation.
Although the amount of calculation was the same, the growth of the rating varied by the way it was used. The Best-of-N (BoN), which generated multiple images and left the best candidate, improved quality in the beginning, but quickly saturated and continuously increased the rating by turning the amount of calculation into intentional reasoning.
Handle editing history and multiple reference images in the same set of instructions.
According to Meta, Muse Imageins the consistency of the image through multiple edits while changing the parts specified by the user.The following instructions can be processed following the previous editing results, so that the image is brought closer to the target while interacting.
The input can use multiple reference images, including people, objects, clothes, styles, and environments, to configure each element into one image. It also responds to instructions that align texts and images with each other, and specifies the reference elements and how they are handled in the same input series.