14 May 2021

Taking A Closer Look At Photoshop's Neural Filters

It's been about half a year since Adobe added "neural" filters to Photoshop. The filters were shown at Adobe Max around the end of October. Let's take a closer look. Calling these filters magic wouldn't be too far off the mark.

It's likely that Adobe will add more neural features, if not in the mid-year update then in the new version that will be revealed at this year's Adobe Max, which will again be a fully virtual event with sessions open to everyone for free.

When released in 2020, the neural filters stack included some intriguing placeholders in the Smart Portrait section, such as the ability to turn a photo into a sketch or a caricature. Skin Smoothing reduces blemishes the way photo retouchers used to do on film negatives. The trick to getting this right is subtlety. Eliminating all blemishes, lines, and wrinkles gives the subject of a photograph a phony, plastic appearance. When done right, the subject looks natural. That's the point of the skin smoothing filter.

The other active neural filters are Smart Portrait, Makeup Transfer, Depth-Aware Haze, Colorize, Super Zoom, and JPEG Artifacts Removal. Possible additions include Photo Restoration, Dust and Scratches, Noise Reduction, Face Cleanup, Photo to Sketch, Sketch to Portrait, Pencil Artwork, and Face to Caricature.

As with most new tools, these can be used well or poorly. In the interest of making clear the power of these tools, I'll be using some extreme settings. But then I'll show an example of a subtle change. The image I'm using is from Australian stock image provider Canva, which also has offices in Manila and Beijing.

The Smart Portrait filter is one that interests me a lot because it offers so many functions. There are sliders for expressions of happiness, surprise, and anger; and there are filters to adjust the subject's age, whether the eyes are looking left or right, the thickness of the hair, the direction of the head, and the direction of the light. The changes can be placed directly on the current layer (a bad idea), on a duplicate layer (with or without a mask), on a new layer that holds just the changed bits (the default), or created as a smart filter.

 Click any small image for a full-size view. To dismiss the larger image, press ESC or tap outside the image.

So let's look at some of the extremes. This is the photo I started with. Note that he is smiling, but his mouth is closed. Moving the slider to the right increases the smile bit by bit until around 15 teeth appear. At the setting I chose (38) the subject has a wide, toothy grin. At the maximum setting of 50, the teeth are the most prominent part of the image. Negative numbers give the subject an increasingly sad look and at minus 50 the expression indicates an enormous amount of disapproval.

I consider 38 to be too extreme. For one thing, the subject will know that the teeth are not his, just something made up by Adobe's artificial intelligence system. The teeth look real enough, and an image like this could be used in a situation that requires a wide grin.

But let's say you've taken a picture of your nephew, Joe. He looks at (1) the photo and says, "I like that, but I wish I was smiling a bit more." You can fix this and give him a (2) slightly larger smile. The (3) happiness slider is set to 8. The most obvious change is the slight upturn of the mouth, but Sensei has made several other small modifications: The eyes are slightly more open and the eyebrows have changed a bit. The jaw line and cheeks have been modified subtly to support the smile. These are all visual cues that people react to naturally. Each of the changes I tried with this image are (4) retained as separate layers that can be activated or deactivated at will.

Each of the sliders has settings from minus 50 to plus 50. The meaning of the slider is determined by the quality being adjusted: Happiness minus 50 is for maximum unhappiness and plus 50 is for maximum happiness. Gaze, head direction, and light direction use minus 50 as left and plus 50 as right. Left and right are based on the viewer's left and right, not the subject's left and right.

Here are some of the extremes I created. It's generally best to avoid the extremes, but I adjusted "Joe" to make the modifications obvious.

Sensei can adjust the subject's eyes (right) so that he's looking left or, as the photographer would instruct him, to his right. I find this effect to be entirely believable, even when nearing the extremes.

"Joe" has a full head of curly hair and Sensei ran into trouble thinning his hair (left). That wasn't a surprise. I also made him older and that was partly successful. The lines around his cheeks look reasonable, but Sensei may have been fooled by hair on his forehead. At the extremes, these two modifications really don't work very well.

But what's amazing is that they work at all. Facial expressions are complex. The American Psychological Association's 2011 article by David Matsumoto and Hyi Sung Hwang explains this complexity and discusses research that has been conducted. The article notes the presence of "micro-expressions", those tiny factors that are important, but nearly invisible. "Reading facial expressions of emotion, and especially micro-expressions, can aid the development of rapport, trust, and collegiality; they can be useful in making credibility assessments, evaluating truthfulness and detecting deception; and better information about emotional states provides the basis for better cooperation, negotiation, or sales."

Sensei does a good job adjusting these micro-expressions, not just the larger and more obvious aspects of expression.

Maybe we can give "Joe" more hair and make him younger (right). Making him younger worked well, but giving him more hair didn't. Changes made in the hair are blatantly obvious, probably because of the amount of hair he already has and the curly texture of the hair.

Turning the head to the left ("Joe's right") is somewhat believable (left), keeping in mind that I selected extreme settings. The head position is fine and the ear modification is good, but once again the hair is a problem. With more modest adjustments, the ear is even more believable but the hair is still a problem.

Turning "Joe's" head slightly to the right worked well, with the exception of the hair, (right) but at the extreme Sensei had to create the subject's right ear. That worked reasonably well, but there's something a bit wrong with the facial geometry and, of course, the hair.

As with all other layers in Photoshop, the layers created by the neural filters can be adjusted. Adding some transparency in the hair or selectively masking parts of the layer could make the hair more believable even at the extremes.

There is, of course, the opportunity for misuse. In part, this is because people generally are not taught to think critically, and even poorly made photo manipulations fool them so long as the image agrees with their perception. "The camera never lies" has never been true. By carefully choosing a lens and an angle, the photographer can create a point of view that has little or no relationship to reality.

Some Civil War photos are composites. During the Soviet era, people who fell out of favor with Stalin were removed from photographs. Many of the manipulations were shoddy and obvious, but some were well done. At that time, photo editing was a time-consuming manual process. Today, some changes can be made in seconds and even complex modifications can be accomplished quickly.

There's no going back, and we shouldn't want to because the technology has so many good uses; but we do need to be more cautious about believing something simply because a photo or a video exists.

Short Circuits

Using Your Senses To Complement Diagnostic Utilities

There are lots of utilities that can diagnose problems with computers, but the first line of defense might be lower-tech involving your senses — sight, sound, and smell primarily. You might sense a problem early enough to fix it by simply paying attention to the computer.

The smell of burning electronics, for example, is a sure sign that something is wrong, but there are other more subtle indications.

<< Some computer technicians say that electronic devices are powered by smoke and that if you let all the smoke out, the device will no longer work.

If a computer that has a mechanical hard drive starts making clicking or grinding sounds, there could be a problem with the disk drive. But this could also indicate a problem with the computer's fan. A fan that's going bad may seem to be a less serious problem, but a failed fan can allow temperatures in the case to rise so much that other components fail. So it's good to figure out where the problem is and fix it before it gets worse.

Listen for changed sounds. If you sit in front of a computer for several hours a day, you know what it sounds like. Any change will be indicative of something — maybe not something bad, but something will be responsible for the new or different sound. This is a good time to break out some of the diagnostic tools I've mentioned before to check disk status, memory health, temperatures, and other conditions.

Noises, in fact, are one of the best indicators that something bad is likely to happen. You should already be backing up at least all of the data on the computer to an off-site location. If the computer suddenly sounds different, make sure that all recent new and changed files have been backed up before investigating what the problem might be.

This kind of general observation is helpful. Let's say the computer suddenly becomes sluggish. It could be a virus. It could be a scammer who's planted malware on your computer to send spam or mine cryptocurrencies. But it might also mean that an application that's running is buggy and is consuming a lot of system resources, or that an operating system update is being prepared.

This is when utilities and diagnostic applications can help. Check the Task Manager to see if some application is hogging the CPU, memory, disk, or network. Try rebooting the computer and, if the problem occurs again, have whatever protective application your computer uses run a scan.

A computer that five years old or older may have insufficient hardware for current applications, a problem that may be resolved by exchanging a mechanical hard drive with a solid-state drive or by adding memory.

When error messages pop up, read them carefully. Write down the exact text or grab a screen shot. Make a note so that you'll remember exactly what the computer was doing when the error occurred. An error message that occurs only once is usually not the sign of a big problem, but if you see the same error repeatedly, it's time to investigate.

Problems range from software errors to faulty device drivers and even failing hardware. The error message often helps to pinpoint the cause. Perform an internet search for the exact error message. Include the text, any ID numbers, and the name of the operating system. Take care with search results, though. Not everyone who provides guidance on the internet is a genius and some crooks use error messages to capture unsuspecting computer users.

Before performing any actions you find recommended online, review responses from those who have had the same problem. In many cases, the best result of an internet search for an error message is a clue about the source of the problem. When you have that, visit the operating system or computer manufacturer's support website.

When you install a new application, it may add an icon to the desktop, but watch for new icons on the desktop or in the Start menu. If you haven't added any new applications, look for the name of the application using a search engine. Malware can make changes to the computer, so scanning the computer in addition to checking out the application can help you decide whether an application should be removed.

Malware is usually installed in a way that makes it difficult to remove, so search for how to remove the application. As with other searches, be especially cautious with this kind of search. The result can lead to malware that makes a bad situation worse. Read carefully to be sure that you're dealing with a reputable operator.

Watch the computer's browser carefully. If the home page has been changed, a "browser helper object" may have been installed. These BHOs can redirect searches, serve ads, and perform other mischief.

Some problems are easily solved, but others are more difficult. When you encounter a problem that you can't fix, your notes and screen shots will be a great help to a technician.

One Unexpectedly Important Computer Accessory

Most of the time I write about computers, applications, screens, keyboards, mice (or mouses*) and the like, but there's one non-electronic component that's important for your comfort and safety.

* Some people feel that "mice" should be reserved for rodents and that the plural for computer devices should be "mouses". I am not one of those people.

Probably the most critical component is the screen because users spend virtually all of their time looking at the screen when they're using the computer. But screens are electronic, so that's not where I'm going with this. Look down. Look past the keyboard.

You're sitting on a chair and the chair is increasingly important if you spend a lot of time in front of the computer. Reasonably good ergonomic chairs are less expensive than they used to be, but you can still spend $1000 or more if you want to. Finding a chair that feels right may take some time, but it's time well spent. Chairs are even more frustrating than printers, but that's not where I'm going either. Look further down.

 Click any small image for a full-size view. To dismiss the larger image, press ESC or tap outside the image.

The chair has wheels. The wheels are probably the cheap kind that don't swivel or roll particularly well. Late last year, I made what may have been my best purchase of 2020: Rollerblade wheels for my office chair. The cheap wheels that come with many chairs don't swivel well, so I decided to try a set of these for $32. The wheels are larger, so I had to slightly adjust the chair height. Ball bearings allow the wheels to turn freely.

And they roll wonderfully well. At least they did until early May when I thought one of the wheels broke, but it was the piece that holds the wheel had broken free from the chair base. The tubular assembly is connected to the chair base with two small spot welds. One weld broke and the next time I tried to reposition the chair, the second weld broke and the wheel assembly fell off.

I had to order a new chair and it took five days for to arrive. Following a significant tipping incident when I forgot that one of the five feet on the chair was missing, I established a temporary workaround with a 1960s dictionary and an out-of-date website development book.

Because it was so difficult to move the chair, I didn't move it, and that led to a reminder about how painful a non-ergonomic setup can be. So if you have back pain, sore wrists, or other symptoms after using the computer, maybe your body is trying to tell you something, and maybe you should take a closer look at the desk, the chair, the keyboard, and all the other items that seem unimportant.

Spare Parts

Tools Needed To Extract Tiny Junk That Computer Components Attract

Keyboard and mice attract fuzz, crumbs, and dirt. We shouldn't eat while using the computer, but how many people do. After a while, the spaces between the keys are filled with crumbs. If you have a cat or dog, you're sure to find bits of fluff in the mouse's scroll wheel.

To remove the junk from under the keys on a keyboard, some people pull the keys off, wash them, vacuum the base, and then put the keys back in. It's possible to do that without a key puller, but it's also easy to break something without a key puller. And removing fluff from a mouse's scroll wheel is even harder. It's possible to take the mouse apart, but good luck getting it back together later.

Some dental hygienist's tools might help. You don't have to abscond with a kit from your local dentist. A complete set of basic dental tools, many of which can be quite helpful in prying, grabbing, and scraping grunge that's lodged in the crevice of a computer device.

You don't need to be a dentist or a dental hygienist to buy the tools, either. I thought they might be available at drug stores, but the only place I could find a set was at Amazon for about nine dollars.

A Computer Chip Shortage May Mean You'll Have A Dirty Dog

Washing a dog seems like it would be a decidedly low-tech process, but it seems like a global shortage of semiconductor chips might result in more dirty dogs.

I ran across this report in the Washington Post, where Jeanne Whalen describes a problem faced by CCSI International, a company that manufactures dog-washing booths. The military buys these units. So do dog parks. Take a dirty dog in and the unit dispenses soap, water, and (optionally) warm air for a blow dry. You get a clean dog without wrecking the bathroom.

But the account says DDSI is one of many businesses that can't get the computer chips they need to build the devices. Computer manufacturers are dealing with problems, too, and auto manufacturers have had to slow production lines because of chip shortages. It's a fascinating account that illustrates how interconnected even seemingly disparate businesses are and how everything depends on computer chips.

Read the full report on the Washington Post website.

Twenty Years Ago: Wondering About Linux While Preparing For PC Expo

I didn't know it at the time, but the upcoming PC Expo in New York City's Javits Convention Center would be the final large event. It wasn't quite as large as the 2000 show, but it would still be an impressive event.

Meanwhile, I continued to wonder when we would see the year of Linux. We're still waiting for that.

I wrote "Linux still hasn't caught on except for business servers and for use by certified geeks. A startup company has plans to put a Linux server in your home."

Linux is a great operating system for servers, I said, and it can work well with Windows. Once you've installed a home network, maybe you need a Linux box to tie things together. That box would be from Memora Corporation. The company calls their box a "personal server" for the home. It offers a residential Internet gateway, a firewall, a wired or wireless network server, e-mail services, and file storage. Linux Journal liked the idea.

Unfortunately, the company seems to have vanished without a trace.