The secret language of AI art
Painter Jessica Helfand uses AI for humanization and dramatization, not optimization.
“The whole purpose of any art form is to let somebody in on a secret.” Jessica Helfand is relaying something a theater director once told her.
Now look at any one of her hyper-realistic portraits. The inner lives of fictional characters, from Icarus to Madame Bovary, are rendered expertly in oil on canvas, with depth, lighting and texture. You can imagine them hanging in the Smithsonian’s National Portrait Gallery – even real people in her commissioned portraits look like they just stepped out of the 19th Century.
When you learn that Helfand uses generative AI and paints on top of photographs, does that change your view of her work?
This article details how Helfand uses AI for humanization and dramatization, not optimization – that’s how she frames it – through an interview about her work and process. In the midst of an artistic backlash against AI, she puts forward a balanced outlook and productive use of AI in the studio. Whatever your views on AI, I think the power and quality of Helfand’s work speaks for itself. And artists will find inspiration and practical advice in her words.
Why AI
Before taking up painting full time about 10 years ago, Helfand was a designer and writer with her own practice, and a professor at Yale working across fine art and design. She has authored several books and has written for notable publications, as well as cofounding Design Observer and many of its related podcasts. Particularly relevant to her current work, she spent her early career writing television soap operas. We’ve known each other since the 1990s, when she wrote about my work for Eye Magazine.
Let’s start with exactly how Helfand uses AI. She uses it to generate characters, not images. For her first exhibition, The Service Society, she researched the lives of Irish immigrants who were employed by wealthy American patrons, many of whom hired John Singer Sargent to produce their portraits. Using language found in letters and other texts, Helfand used these in her prompts – thus injecting a kind of primary source language into her work – and then collaborated with Midjourney to both iterate and (partially) generate ideas for portraits.
In her second exhibition, Character Studies, she entered descriptions of the characters lifted directly from literature, the language drawn from the authors who created those same characters. Emma Bovary was the first.
“There was a section in the book in which Flaubert describes the exact moments between when she takes poison and when she dies,” Helfand tells me. “Her eyes roll back in her head, her lips get parched, her skin turns blue. And all of these things are happening at once.”
“And so I plugged that in and, Oh my God. I was so excited. It didn’t give me exactly what I wanted, but combined with where I wanted to go, it was just a kind of life-changing moment for me in the studio.”
“You know,” she goes on, “we stand on the shoulders of giants. Why would you not try to view AI as a way to build upon something that’s preceded you?” She often paints a portrait twice, because she doesn’t like it the first time.
“I mean, to not work with photography just ignores the primary image-making method of our generation. I’m collaborating with an engine that helps me mobilize my thinking visually, towards a form of portraiture in which I am not the primary voice.”
Helfand painted on photographs well before starting to use AI, and over time, she has continued to refine the process. “Today, I work with a master printer. I worked long and hard to find him, and he understands me and what I am trying to accomplish. He prints a sketch on canvas for me, in archival inks that are light resistant, lightfast, beautiful. They have an exquisite, buttery texture to work with, and working over them in oil helps me make these paintings which have a kind of soft, deep luminosity.
“I make paintings that look like they were done more than 100 years ago,” she adds, “but they have solid backgrounds, geometric compositions, and a kind of implicit modernity. And where AI is concerned, they are absolutely informed by language.”
Crowding the studio
In the history of human image-making, from cave paintings onward, some of the best pictures are stories. Some might be used alongside written or narrated stories, while for others the title is a ticket to a narrative journey. In both cases, language can be just as important as the image.
For Helfand, that language is mostly front-loaded: in the stories that inspire the paintings, and in the prompts that help generate them. But it also comes afterwards. This past year, she published a kind of working studio diary on Design Observer, sharing the experimentation, the research, and the discoveries along the way.
When it comes to creativity and meaning making, language is just as much a tool as a paintbrush or Midjourney. Psychologist Lev Vygotsky called it the tool of tools.
“I’m not looking for a bot to make a picture,” Helfand notes. “But I am using technology in a very specific way to help me mine different kinds of information, and the essential thing for me here is how I use language. Sometimes I’ll look at other people’s Midjourney prompts; I’m interested when, for example, they use a word I’ve never used before. And I’d say that for me, in general, the prompts people create are probably more interesting than the images they actually produce.”
Helfand’s experience as both designer and writer makes her more than a fine artist. “This is not a painterly skill. This is an editorial skill. Maybe it’s because I’m old. Maybe it’s because I’ve taken so many pictures of my own children over the years. But I know how to fix photos. And if those photos are the source material for a painting – helping to draft the underpainting, in a way, then this is an important phase of the sketching process.”
“I think ‘director’ is a word we haven’t used yet,” she continues. “As an artist you are your own director. You’re directing the paint. You’re directing the composition. You’re directing the premise, the tone and mood, the idea behind the painting. And, naturally, you’re directing the central players.
“When I was studying theater, a director once said to me: ‘At some point in the rehearsal of a play, you will know more about Lady Macbeth than I do, and I have to back off. I have to trust that you’re inhabiting this character, you own it now.’”
It’s there in the staging, the lighting, the imagined back story of a character. “My children refer to my paintings as my ‘imaginary friends’,” she says. “Some evenings, I feel like I have to go back into the studio and say good night to them all.”
She relays a quote from painter Philip Guston: “When you’re in the studio, there are many people there with you – your parents, teachers, friends, critics, buyers, your gallery. One by one, if you’re really painting, they all walk out. If you’re really painting, you walk out.”
This is akin to how a novelist’s characters take on a life of their own. Artist Ian Cheng refers to “worlding” – creating a world that can exist after you, the creator, walk away. For novelists, it often means creating compelling characters and then placing them in difficult situations. For Helfand, it can mean taking pre-existing fictional characters and giving them a new life. This is how her most recent series works.
AI as wax wings
She became interested in Greek mythology, and latched onto the story of Icarus, the boy with wings who flew too close to the sun, with disastrous results.
“This is a story that’s universal across cultures,” she says. “In China, Eastern Europe. It’s something that everyone knows. I became interested in how this myth transcended popular culture, and I looked across a variety of media: poetry, for example, and song lyrics.”
“Also, Icarus’ father was a designer,” she adds. He created the wings, after all. “And kind of a difficult person. He resented his nephew’s talent so much that he threw him off a roof.”
Might the story also serve as a kind of metaphor for AI? I ask.
“In the sense that greed is an international epidemic,” she replies. “I can paint something that maybe underpins the idea that we all think there’s some brass ring waiting for us. And we’re willing to damage ourselves, to endanger our relationship to others, in order to try and cheat some fate that we’ll never actually see.”
The story took her down many rabbit holes – exploring the human desire to fly, to create. Icarus as a child, a teen, an adult; maybe even as another gender.
“I wanted to look at peripheral sources, anecdotal ways that might reveal how the myth of Icarus has seeped into our collective cultural consciousness. The first two collections I made used the language of AI to push the image, but this one did not. It’s been more impressionistic, looser, more experimental.
“So the more I deconstructed this myth, the more I found stories that were not necessarily about flying too close to the sun – but they were all about flying too close to the son. That relationship is everything.”
Around the time of our interview, the Pope released his encyclical about AI. “Who was with the Pope?” she muses. “A guy from Anthropic. Maybe that’s our modern-day equivalent of the myth. The killer app, the viral video, crypto – some pot at the end of some rainbow that maybe AI is going to help us find.”
“This might be the opposite of influencer culture,” she adds. “Maybe AI, when we’re not looking at its nefarious potential, is an incredible companion for that dialogue. It’s not sentient, but it is a kind of stimulant.”
The secret’s out
“My paintings are not going to down the government or upend the economy. But, I have a profit margin of my own, because the paintings are selling. Knock on wood. I’m happy about that. Before I started painting full time, I was not doing as well, financially, as a designer. Mostly I fel like what I’m doing now is much more legitimate and honest and real. At first, calling myself an artist felt pretentious, but now it just feels accurate.
“I like to think that artists are not all doomsday about AI. I mean, the things you’re doing and writing about, the contact we both still have with students – I think it’s important that we ask penetrating questions about what constitutes practice, and philosophy – and machines, right? What constitutes the intersection of the world we have inherited with the world we’re leaving for our children?
“What artists have, that no one else has, is the capacity to imagine, and share that imagination with others.”
If art is about letting the viewer in on a secret, AI doesn’t seem to have a place there. “AI doesn’t keep secrets,” Helfand says. “It’s all out there.”
Want more? Read an edited transcript of our interview here. Jessica Helfand’s website: https://www.jessicahelfand.com