Using impossible software to create impossible worlds – the rise of AI and the art of digital painting.

This article was commissioned by Dazzle Pictures, and appears in their website.

AI, or Artificial Intelligence (with capital letters), sounds like something out of a science fiction movie – like the omnipotent and somewhat annoying trans-dimensional being known as Q in Star Trek, who was known for meeting out unintelligible and incomprehensible wisdom from beyond. In reality, AI is very real, and it’s everywhere.

You can’t click onto a website, open a magazine, or scroll through your socials without bumping up against it. Your phone uses a form of AI to unlock everything from its lock screen to your banking apps. Tools on your computer are helping you to write error-free emails and documents. The ads that track you across the web are powered by AI. That voice thingamajig that you use to speak to your home devices – yep, that’s AI too. There are even apps that claim to be able to write copy, and, presumably, articles like this one. Thankfully this article is being written by an actual human being even as you read.

And in case you were wondering, we’re not going to launch into a long-winded explanation about what AI really is – frankly speaking that’s for a whole other article. The term itself comes from the idea that machines can be given the ability to perform similar cognitive tasks to human beings – doing so via a neural network that romantically follows the architecture of the human brain, where nodes in a circuit are compared to neurons. It’s heady stuff indeed.

Suffice it to say that for our purposes we can define AI as computer systems that are able to look at enormous quantities of information, apply algorithms to this information and then produce novel artifacts based on these underlying datasets. In other words, systems that look at everything that has been created or produced before, and use this information to create something new. 

Like digital art for VFX studios and other creative endeavors.

And it is here that we come to the starting point for this article.

Perhaps you’ve also noticed this, but recently, we’ve been hearing a lot of buzz around the topic of AI art – that is to say, imagery that has been magicked up, entirely out of thin air using only an AI app (of which there seem to be exponentially more examples each day). 

As is often the case with any new technology, the use of AI in art and design has fractured the world into various conflicting camps.

On the one hand, there are those who see this as a veritable death knoll for artists, and indeed anyone who makes their living in the visual arts. Just like those who viewed the advent of the digital camera as the end of photography as we knew it, there are the doomsday nay-sayers who woefully predict AI is on its way to putting us all out of a job.

Some point to the amount of effort at the input side as being too low when weighed against the incredible sophistication that comes out at the end. The argument is that this is simply unfair, somehow. That people are making art without doing the work.

AI is a tool, just like any other.

The counterargument is that AI is just a tool like any other. Think of technologies like Photoshop – which has its own built-in AI called Adobe Sensei, and which has reduced the work needed for things like creating alpha channels and selections exponentially, as well as the relatively new Neural Filters which can do anything from re-colouring old photographs to replacing skies literally at the click of a button; and Autotune– which has been helping singers sound better since 1996, and has even gotten certain rappers into trouble. Google it.

The point is that such technologies are only there to enhance what you already bring to the table in terms of talent; to wit Photoshop still requires a huge amount of effort to create anything original, and as for Autotune, well, you really do have to be able to sing at –at least at some level. Watch the American Idol auditions if you doubt us.

We are not in the naysayers’ camp.

As a matter of fact, we love it when a new thing comes along – along with the potential downsides (often not really downsides) new technology always brings with it extraordinary advantages. 

And it’s on those advantages that we want to focus. 

Now, before we go any further, we should point out that this article is not about the ethics of AI, how it affects society, or even how it might, or might not, change our definition of art and/or creativity; this is, quite simply, about using AI as a tool like any other in our arsenal, and how AI-generated imagery fits in with our particular workflow.

We argue that the use of AI in the visual arts is about more than just getting as quickly as possible to an end result.

AI is fast becoming, if it is not indeed already, an essential tool in the VFX workflow – specifically, the area of matte painting. 

At its core matte painting is about creating the impression of an environment that was either not present at the filming or too expensive to film for real, or simply impossible to shoot for one reason or another. A matte painter, or matte artist, paints a – usually static – background, or other parts of the film world, in order to produce the illusion of something that looks real, but that in fact does not exist in reality. In one case this might be the inside of a giant star freighter, while in another it might be the smoky backdrop of H. G. Wells’ industrial London. 

Matte paintings don’t necessarily have to look real, they just have to match the look of the rest of the film. Matte painting’s job is to complement the look-or concept–development aspect of the film.

Matte painting has been with us since the very beginnings of moviemaking. We’ve all seen the famous death-defying Charlie Chaplin skating scene in the 1936 film Modern Times, where Chaplin skates around an upper floor in a department store, coming perilously close to the edge of a sheer drop where the railing appears to be broken. The effect, of course, was created by placing a matte, painted onto a piece of glass, between the camera and the action. 

What seems so simple now in fact represented the pioneering use of technology at the time. So influential did this technique turn out to be, that it became central to the special effects used many years later in a very famous series of movies set in a galaxy far far away. 

In the digital era, traditional painting techniques have been replaced by digital compositing methods. And although the basis for creating these matte worlds may have evolved with the advent of new technology, the principles remain the same – with VFX artists creating extensions to the worlds being presented in the live-action sequences. Creating, in essence, what is not there at the time of the shooting. This might be a representation of medieval Prague, as seen from across the Charles Bridge, or the Emerald City in the Wizard of Oz – matte painting runs the gamut from the perfectly realistic to the utterly fanciful. 

AI technology allows us to do what we have always done – create conceptual worlds and environments – only in a far more efficient way than was possible before. The craft of what we do is not diminished – Wacom tablets and photo-bashing in Photoshop are still essential tools and skills, but now we have another means to help us create the building blocks. And as the building blocks get more sophisticated with every new iteration of whatever AI software is the app de jour, so too, does our work.

And the better our work gets, the more the AI has to train itself on. It’s a kind of meta-spiral that’s sure to break your brain if you think about it too hard.

Finally, if there is a conclusion to all of this it is that intent counts. Just as when you see a work of art in a museum that speaks to you in some kind of way, if what you are creating using AI helps to tell a story, then perhaps it is not the great evil some are making it out to be.

AI is here to stay, and it’s only going to get more sophisticated. And we’re on board with that.

---

---

---