One of the most interesting AI application developments in recent times has been Dall-E, an AI-powered tool that lets you enter any text input, such as “horse using social media,” and will generate images according to the your understanding of this data.
You’ve probably seen many of these visual experiments floating around the net (‘Wird Dall-E Mini Generations’ is a good place to find more unusual examples), some of them very useful and applicable in new contexts. And others are just weird, distorting interpretations, showing how the AI system sees the world.
Well, you might soon have another way to experiment with interpreting such AI, using Meta’s new “Make-A-Scene” system, which also uses text cues as well as input drawings, to create completely new visual interpretations.
As Meta explains:
“Make-A-Scene allows people to create images using text cues and sketches freely. Earlier image generation AI systems typically used text descriptions as input, but the results could be difficult to predict. For example, the text entry “a painting of a zebra on a bicycle” may not reflect exactly what you imagine; the bicycle may be looking sideways or the zebra may be too big or small. “
Make a Scene aims to solve this, providing more controls to help guide your exit, so it’s like Dall-E, but at least from Meta’s point of view, a little better, with the ability to use more directions to guide the system.
“Make-A-Scene captures the design of the scene to allow nuanced sketches as input. It can also generate its own design with text-only indications, if that’s what the creator chooses. The model focuses on learning aspects key to images that are most likely to be important to the creator, such as objects or animals “.
These experiments show the extent to which computer systems have come to interpret different inputs and the extent to which AI networks can now understand what we communicate and what we mean in a visual sense.
Finally, this will help machine learning processes learn and understand more about how humans see the world. Which might sound a little scary, but it will ultimately help fuel a number of functional applications, such as automated cars, accessibility tools, augmented reality and enhanced virtual reality experiences, and more.
Although, as you can see in these examples, we are still a bit far from AI thinking as a person or becoming sensitive to their own thoughts.
But maybe not as far as you might think. In fact, these examples serve as an interesting window into the ongoing development of AI, which is now for fun, but which could have important implications for the future.
In their initial rehearsals, Meta gave several artists access to their Make-A-Scene to see what they could do with it.
It’s an interesting experiment: the Make-A-Scene app isn’t available to the public yet, but you can access more technical information about the project here.