Smart Homes – Challenges in Designing Experience Beyond the Screens

Smart homes are becoming more and more popular as we speak! Just by looking back at the past decades, we can see how the interaction between people and smart devices has changed drastically from early computers with text-based commands to graphical user interfaces, all the way to mobile and touch devices. And the evolution of technology doesn’t end here as today we’re experiencing the integration of IoT services into our homes controlled with our voice. So let’s see how this new era affects us as visual designers and what sort of challenges are we facing in designing experiences beyond screens.

Looking back at sci-fi movies from the 80s and 90s, they weren’t that far off from how technology in future homes will look like.

sci-fi movies and smart homes

The smart home product market has expanded. Nowadays you bump into the terms ‘smart home’ and ‘home automation’ everywhere, but what do they mean? What is your experience with smart homes? Do you consider having one?

What is a smart home ?

Your home is considered to be “smart” through a collection of smart devices connected to a network that can be controlled remotely or by voice-controlled smart home hubs which replace direct interaction with your household. Such devices can be found in the kitchen: smart fridge, smart microwave, smart oven, you name it, which can all be monitored with your phone. Programming a thermostat or an AC has never been easier, as they too can be scheduled through an app. You can monitor and set the temperature while you’re out.

No need to panic in case you forgot to lock the door, you can now do it through an app. You can also check on your pet through a smart cam to make sure everything is okay. You can basically have control over almost everything you own.

Google Nest Hub Max
Google Nest Hub Max

Technology is starting to adapt to how people normally communicate. With voice-controlled interfaces, users won’t have to go through the usual learning curve that traditional web forms have, to learn and master the use of an input device. Not having to learn how to use an interface first, results in a smoother, improved user experience.

Does your smart home understand you?

I still have to add the fact that although communication is a natural action to people, adaptation is slow. Even though there are the early adopters who have set up their homes with Google or Echo devices and couldn’t imagine their lives without a smart kitchen, smart security and smart climate control, the majority, including myself, are still getting used to the way voice is integrated in our home.

I’m still skeptical about using my voice to ask a device to do something instead of using a screen. My colleague wrote an article about voice-enabled design in which she expresses her concerns on how devices are struggling in recognising commands from a non-native English speaker.

Does your smart home understand you?

Regarding other nationalities, as designers, we should be aware of the fact that smart homes will be different depending on cultural differences. The technology and preferred products may differ depending on which part of the world the target audience lives. Thus the tone of voice should be different depending on demographics.

What to look out for as visual designers

This is a great opportunity for us, designers, to transition from screen to voice and to start creating products that are much easier to use, to make people’s lives easier by changing the way they perform everyday tasks. Not to mention spending more time with other people rather than interacting with all sorts of devices.

So how can we design smart products that improve users’ wellbeing at home? And what additional competencies should we have in order to keep up with the evolving tech?smart home designs

  • For starters, we have to find out the users’ needs and give them a solution within the overall design. Think of it this way: the user has a problem, what kind of solutions can we create to solve it?
  • Use cases can help define the ways users might interact with a smart device.
  • Besides the usual team, other professionals such as electricians or architects might come into the picture whom you should consult the feasibility of the product within the home.
  • It should be an iterative process, but it should be considered that changes on a physical object might be more expensive, that’s why the discovery phase should be very thorough.
  • Defining the personas is a good way to find out the tone of voice of the product.

The best smart home devices will hear what people say

But there are multiple conditions that have to be considered when defining the way people will speak to the device.

best smart home devices

As mentioned earlier, the users might have an accent, or speak from far away. They might get interrupted or might cough during a command. But there could be many other similar factors that can challenge the device’s understanding of what the users meant. Not to mention the countless ways they can express one command. There could be a very technical persona who would tell a command in great detail whereas another could be more vague.

The device should be able to execute the action even if the user didn’t give all the information that was needed to do so. It has to draw a conclusion from the context. Try to understand the intent then confirm it with a number of questions. So instead of making the user repeat the same command, make the device ask for the missing information instead. This will result in a more natural conversation. Also, the AI should let the user know that the command has been confirmed and executed.

Thinking like a user is key for a smooth experience

It is very important to define how they would like to get from A to B.

Himel's voice-user interface design
Himel’s voice-user interface (VUI) design

Building up as many flows as possible of the whole process between the user and a device takes time and it shouldn’t be rushed. The device will fail multiple times but that’s totally okay, this is the way it will be improved. 

So when defining the structure of speech, the main things to consider are the following: what people say, what they actually mean and what the result should be.

Thinking like a user is key for a smooth experience

After defining the persona and all their potential ways to communicate, the device’s persona should be tailored accordingly. Is it friendly? Does it match the company brand? Does it sound pleasant?

Moving on with the competences, designers should also have a solid knowledge of system design and awareness of consistency. If the designed product has a built-in Alexa for example, it should follow the guidelines to keep consistent experience with what people are already used to. Unless user tests prove otherwise.

Even though the main interaction with smart devices will be based on voice, they need to have a screen or some sort of visual indicators. The best way to decide whether the device needs a screen or if just a simple LED light will be enough, is to test it with users.

Communication should feel natural with smart home products

People are used to getting feedback, seeing a reaction when talking to someone. Thus smart devices need to show somehow that they can hear the user, or indicate that they are waiting for a command or processing it.

smart home productsAnother important factor is being able to turn off the device’s microphone, as most people are very concerned about their privacy. They still feel uncomfortable by having a device in the house that hears everything. This might be one of the reasons why adaptation is so slow. So having an indicator that shows when the device is turned on is one of the most important features.

Voravit Vorapanyasakul - Google Home Hub Redesign Concept
Voravit Vorapanyasakul – Google Home Hub Redesign Concept

If the product has a screen, it is important to show the main functions of the device rather than the built-in voice AI’s functionality. Another important factor is to separate the visual controls from the voice experience. Also, keep in mind that people might stand far from the device so the elements on the screen should be of a size that can be easily viewed from a distance. This means that it shouldn’t be overcrowded. It should be clean and to the point containing only the necessary elements. Here’s an interesting case study about Google Home Hub’s visual experience.


Remember, at the end of the day, smart homes should be all about solving people’s problems and making their lives easier. To save their time by either providing them hands-free communication – not having to handle the device while doing an activity that requires both hands – or to execute commands remotely, like heating up the place before getting home. And last but not least, empathy is key for user-centered design. Putting yourself in the user’s shoes can result in much better products.

This is still just the beginning of the era of smart homes. Voice-based devices still have a long way to go to reach a natural level of conversation. 

As the market matures, people will end up integrating more and more products into their homes. To maintain the harmony and the sole purpose of making life easier, these devices should be able to communicate with a hub instead of having their own individual app.

Do you have any smart devices at home? Let me know what you think about this whole new experience. If you’d like to read more about voice user interface check out this article we wrote.

Continue learning with UX studio

Here is our fresh interview “We’ll Need More UX Writers in Tech” –  with Yuval Keshtcher (ux writing guru). Make sure to read!

For additional reading, check out our Product Design book by our CEO, David Pasztor. We ship worldwide!

Andrea Fabian

Designer, dog addict, proud owner of a telescope, Master's Degree in dad jokes.