Neural Style Transfer, Running on Your Device

Neural style transfer takes the arrangement of a photograph and the visual vocabulary of a painting and produces an image that has both. It is one of the few genuinely interesting things a neural network can do in a few seconds inside a browser tab, and our AI style transfer tool does exactly that - the model runs on your machine and your photo is never uploaded.

Content and style, separated

The insight behind the technique, published by Gatys, Ecker and Bethge in 2015, is that a convolutional network trained for image recognition ends up with two separable kinds of information in its activations. The deeper layers encode what is where - a face here, a horizon there - largely independent of colour and texture. The correlations between feature channels within a layer encode how it looks - brushstroke size, colour relationships, repeating texture - largely independent of layout.

Style transfer optimises a new image so that its deep-layer content matches your photograph while its channel correlations match the painting. The output is not a filter applied to your photo and it is not a copy of the painting. It is a third image that satisfies both constraints as well as it can.

Why this one runs in a browser

The original method solved that optimisation from scratch for every image, which took minutes on a GPU. What made the technique practical was arbitrary style transfer: a style prediction network converts the painting into a compact style vector, and a transfer network applies that vector to your photo in a single forward pass. No per-image optimisation, no server round trip.

The tool loads TensorFlow.js and Magenta's arbitrary style transfer model from a CDN the first time you open it, then runs everything locally. The download is the model weights coming to you, which is the exact inverse of the usual arrangement where your photo goes to the model. Nothing about your image is transmitted, and there is no queue, no credit balance, and no output watermark.

Why the output is not huge

Style transfer cost scales with pixel count, and a browser runs it on the main thread of a tab that also has to stay responsive. Stylizing a 12 megapixel phone photo would lock the tab for a long time or run out of memory outright. The tool therefore downscales the content image so its longest edge is around 500 pixels before stylizing, and the result you download is at that scale.

That is a real constraint, and it shapes what the output is good for: avatars, social posts, header images, prints at modest size. It is not a route to a poster. If you need more pixels, the honest sequence is to stylize first and then enlarge the result with an upscaler, accepting that the brushstrokes get enlarged along with everything else.

Choosing a photo that stylizes well

The network preserves large structures and rewrites small ones. Photographs that survive that trade have a few things in common:

• A clear subject with strong shapes. Portraits, buildings, single trees, boats on water. Anything that reads as a silhouette will still read afterwards.
• An uncluttered background. Fine detail becomes texture, so a busy background turns to noise rather than to art.
• Good separation of light and dark. The style's palette is mapped onto your luminance structure, so a flat, low-contrast photo produces a flat, muddy painting.
• Deliberate cropping. Crop before you stylize, not after. The downscale step means every pixel you were going to throw away was competing for the limited resolution you get.

Faces are the classic hard case. Style transfer rewrites exactly the fine detail that makes a face recognisable, so expect the result to read as a portrait rather than as a likeness. Tight head-and-shoulders crops fare better than group shots.

Choosing a style

The tool ships three well-known reference works, each with a very different signature. Van Gogh's The Starry Night contributes swirling directional strokes and a blue and gold palette, and it dominates whatever it is applied to. Munch's The Scream brings smeared, wavy bands and an anxious orange-and-teal cast that suits sky and open space. Hokusai's The Great Wave yields flatter forms, hard outlines, and a restrained indigo palette that reads more like illustration than painting. Try all three on the same photo - which one works is a property of the photo, not a ranking of the paintings.

Afterwards

Stylized output is a PNG, so it drops straight into the other tools. Round the corners for an avatar, or pull a palette out of the result to build a matching design system around it. The colour side of that workflow is covered in extract, build a palette, adjust, and rounded corners for avatars and icons handles the framing. More articles are indexed on the Privacy Pix Tools blog.