go back
1

Creative tiling: Artomatix applies A.I. to smooth out textures


By 3dtotal staff

Web: https://artomatix.com (will open in new window)

|
(6254 Views) | 1 Comments
| Comments 1
Date Added: 19th December 2016

Dr. Eric Risser, founder and CTO of Artomatix, offers his take on A.I. in the 3D creation process and what it will mean for artists of the not too distant future…


937_tid_inftile.jpg

937_tid_zArist-Profile.jpg

3dtotal: Hi Eric, please tell us a little about yourself.
Eric Risser: My name is Eric and I'm the CTO of Artomatix, an Irish company just down the road from Havok and Demonware. Here at Artomatix we specialize in Artificial Intelligence applied to creativity. We've developed a new way of making 3D art that automates the most tedious and mundane of everyday tasks. This way artists can spend their time focusing on the fun, creative aspects of a project, not the grunt work. We call this new approach, "example-based art creation.”

3dt: What made you step into the videogame realm?
ER: I've always loved videogames. My earliest memory is playing Super Mario Bros for the first time. Twenty years ago, I rigged my Sega Saturn for imports and got a first-run edition of Radiant Silvergun. I used to measure success in life by how many games I could beat before I die. Obviously, kids grow up and priorities change, but I can honestly say that I didn't step into gaming, it's always been a part of who I am.

3dt: Why did you choose to become a doctor of A.I. art creation?
ER: Well, I suppose I just kind of fell into it. When I was working on my bachelors in computer science, I was offered a job in my university's computer graphics lab as a research assistant. At the time I was planning on becoming a videogame programmer so I figured it would be a good chance to gain some expertise. When I started, they told me that I needed to choose my own "research problem” and it had to be (1) important, (2) difficult and (3) unsolved. I'd always been interested in A.I. and I knew from working on indie games that creating all the art was a huge challenge, so I thought it could be interesting to work on that problem. More importantly at the time though, it let me justify my $10/hour job.

I stuck with the project and a decade (plus three college degrees) later I found myself as a doctor in an incredible niche yet valuable field where there were only a handful of other experts – A.I. for art creation.

937_tid_zzzIT-1.jpg
Conventional tiling creates obvious seams

3dt: A.I. seems to be in the news a lot these days. Why do you think A.I. is gaining so much momentum compared to other emerging technologies?
ER: It's true, Artificial Intelligence is a very hot topic right now. I believe there are several factors that contribute to this:

(1) The internet and the "internet of things” has produced incomprehensible amounts of data. A.I. is the only tool capable of extracting a signal out of all that noise, so industry and government are both deeply invested in A.I.

(2) Deep Learning (which is the new buzzword for Neural Networks, a sub-category of A.I.) has recently transitioned from a 50-year long science fair project to a breakthrough technology that is achieving state of the art results rivalling humans in several fields. A big part of this recent success is due to the availability of big data sets (thanks again to the internet) that can be used for training, and GPU's becoming general purpose enough that they are now capable of running Neural Networks orders of magnitude larger and faster than was previously possible. Nvidia in fact has put considerable resources over the past five years into turning graphics cards into Neural Network super computers. Thanks to all of this, things like self-driving cars, which would have been science fiction five years ago, are now a reality.

(3) I believe the most important reason why A.I. is taking centre stage is due to its wide application in so many fields. Like computers, or the internet, A.I. isn't a technology that's been designed around a single purpose or application, it's a general-purpose "enabler” technology that can enhance and improve almost anything from biotech to architecture. Chances are that cancer and cold fusion are going to be solved with the help of an Artificial Intelligence in the future.

On that note, Artomatix is applying A.I. to the creative industry, a field that has never utilized it before, and we're seeing some amazing things as a result!

937_tid_it-2.jpg
Infinity Tile covering a large landscape in a non-repetitive way

3dt: Functionally speaking, how do you see example-based art creation affecting the game industry? Why should people be excited about this?
ER: Well, videogame developers have a problem; art creation costs too much and takes too long, especially with the increasing demand in both quantity and quality. 3D artists are super passionate about what they do and are problem solvers in their craft. We're trying to solve problems for the problem solvers, but on a huge scale.

I can't say too much about what the rest of the industry is doing. I know that EA is setting up Frostbite Labs to start building their knowledge base in this area. I also know that several game engines are pretty keen on these technologies. My prophecy for example-based art creation in the game industry however is the one I'm actively trying to fulfil through Artomatix. People should get excited about this new generation of tools because it will make their jobs easier and more fun. I mean the whole point of machines is to do our tedious and gruelling work for us, so that's what we're trying to do for art, cover the boring and mundane tasks so artists can focus on the fun and creative work… I suppose I'm still being a bit vague. I'll explain what I mean in detail.

Initially Artomatix is fixing the pain of material authoring. Let's say you've found an image on the internet that could make a perfect texture for your game, except it's the wrong resolution and it doesn't tile. That's the sort of time consuming and tedious job I've been referring to, the sort of job you wish you had an assistant to do for you, that's where Artomatix comes in.

937_tid_Seam-removal-3D-Total.jpg
Artomatix seam removal in action

Tiling texture is a staple of 3D graphics, but sometimes the tiling artefacts are too apparent and you'd prefer the same texture, but a larger version of itself with more variation.

937_tid_Texture-mutation-example.jpg
Artomatix mutates (or grows) a new variant of the original input

In fact, sometimes your source texture might have artefacts in it, or just some content you don't really want. Imagine if you could just tell your computer to get rid of it for you, like content aware fill, but for PBR materials.

937_tid_Ignore-mask-example.jpg
Ignore mask removes unwanted features

You can grow out textures, but ultimately there will still be tiling artefacts. This is where procedural techniques usually come into play, where tech artists write programs to make their textures appear infinite. We've also built an example-based/procedural hybrid approach we call Infinity Tiles that essentially does all this work for you, turning any material into an infinite version of itself without really affecting your memory footprint.

937_tid_IT-example.jpg
Infinity tile example

But this is really just the beginning of what we can do. The era of Photogrammetry is around the corner and the industry is already starting to feel the pain due to a lack of tools for this new workflow. Once you scan the world, there's a lot of tedious clean-up to do, removing artefacts, seams, adding variation, and so on. Example-based art creation is actually the perfect technology to complement Photogrammetry.

937_tid_Up-Res-example.jpg
Original image resolution improved x 4

Without delving too far into it, we're always researching and developing new technology that will change the way people make games. Rather than making everything from scratch, artists should start from reference material, either grabbed from TurboSquid or one of the many content repositories out there on the web. The reason why these repos aren't commonly used for production is because the assets are extremely heterogeneous in terms of both quality and style. We've been developing technologies to improve both. Essentially, we can take old, low resolution assets and imagine what they'd look like at a higher resolution, using an A.I. that learned how to enhance textures by looking at terabytes of examples. The other big problem with grabbing your assets from a store, or even photogrammetry, is that they probably won't perfectly match the style, or the look of your game. We've been working on that problem too using systems that can learn and transfer your own artistic style onto any other content. Basically, you could grab a character or prop from TurboSquid and automatically re-imagine it to fit the style and quality of your game!

937_tid_Style-transfer-2.jpg
Artomatix team member in Picasso style

In the longer term, example-based art creation will go way beyond just cleaning up your data, it can actually help you create new things and explore a creative space. The hybridization technology we've developed can take a few examples of anything as long as they all belong to the same category (e.g. zombies, dogs, shoes, whatever) and create infinitely more.

937_tid_Populate-image.jpg
Artomatix's algorithm can create countless unique variants from just a few inputs

The icing on the cake though – we can extend all these image-based techniques to work directly on 3D models. But really the list is potentially endless, we just need to know where to start next.

Related links

Check out Artomatix's website
For more from Eric check out his website

 
1
Related Items

Interview

Creative tiling: Artomatix applies A.I. to smooth out textures

Dr. Eric Risser, founder and CTO of Artomatix, offers his take on A.I. in the 3D creation process and what it will mean for artists of the not too distant futur...

Go to galleries 1
Comments 1 Views 6255

Interview

The 10 most popular articles from January to March

Check out our round-up of the most popular tips, tricks, interviews and reviews from the last 3 months....

Go to galleries 1
Comments 1 Views 10156

Interview

10 top 3D texturing tips

In the second part of our Top 10 series, we explore the wonderful world of texturing to find out what expert advice we can take from industry pros for our own w...

Go to galleries 1
Comments 0 Views 38088

Interview

Artomatix: the brush that thinks for you?

Artificial Intelligence pioneers Artomatix discuss how their tools can help innovate some of the more mundane tasks present in the 3D industry…...

Go to galleries 1
Comments 0 Views 3800
Readers Comments (Newest on Top)
avatar
David Edwards on Tue, 20 December 2016 9:28pm
You should move into intelligent / procedural world building. I think as user expectations of VR grow, and budgets continue to shrink, the need for creating world size assets, such as a cityscape modeled down to every door knob will become a big demand.
Add Your Comment