How to design a survey/questionnaire to evaluate user appeal of a visual design?
There has been a lot spoken about user experience design for interactive products and services, but is there some standard or guideline to capturing user feedback for non-interactive products that has a significant visual design element to it?
I think the difficulty is how to evaluate the different elements of design that contribute to the aesthetic appeal/quality, so the indirect way to measure this is to understand what the users think. Has anyone tried to create some systematic way to evaluate this?
This article discusses some of the principles of visual design, and I can see elements of it being adopted for a survey or questionnaire focusing on each aspect of:
- Consistency: Do you notice things that don't fit well together?
- Alignment: Do things feel nice and neatly grouped?
- Proximity: Do things feel logically grouped together?
- Contrast: Are things easily noticeable and easy to read?
- Hierarchy: Are things in the order that you would expect to find them?
Update: I saw this paper that goes into some detail about the study of website design appeal which I think some people might find interesting. It points out some general trends between different demographics that are worth noting:
- females liked colorful websites more, and colorless websites less, than males.
- both genders reached their peak appeal at a similar low to moderate complexity level, but females disliked simple websites more.
- adults aged 41 years and above liked websites with a higher colorfulness and complexity than younger age groups.
- negative correlation between education level and colorfulness, as well as between education level and complexity. Independent of age, highly educated users prefer less complex and less colorful websites than others.
- a userâ€™s geographical location is an additional factor influencing appeal.
Surveys and test marketing have been around since at least the 1930s, for good reason. Any time a company is about to spend a large amount of money on a marketing campaign, movie release or product launch, there will be surveys and testing. The bigger the investment, the bigger the risk, the greater the detail of surveys and/or extent of the tests. Poor or no surveys = poor or no results and a waste of money. There are many books on the subject, and any marketing course or text has extensive information on surveying.
Whether it's usability testing, test screenings of movies (with survey of the audience afterwards), reaction surveys, focus groups or in-person interviews, the only way to know how people will react to a given piece is to survey (if it's something new) or find out what has already been shown to work for that audience, preferably both. Sometimes you'll find the client's marketing or sales staff have a wealth of information about what works for their audience. Never ignore that; they know their public much better than you do.
Graphic design never exists in a vacuum. It always has a purpose, and most of the time that is a marketing purpose. Aesthetic appeal far from the only criterion. Those hideous-looking used car ads you still see in newspapers and online work when they're done like that. They don't work when they're aesthetically designed. (You may take a moment to recover from the horror of that before continuing. We understand.)
Before you can design something, far less decide what survey questions to ask or how to test, you have to know what the design is supposed to do. Your own evaluation of what works or doesn't is the first cut, then you go into the field and find out, using surveys or testing, how your audience reacts or responds.
If it's packaging, put it on a shelf with competing items and ask people which they would be likely to buy and why. If it's an ad, let people look at it for a few seconds, then ask what they got out of it. If you're typesetting a book, set up a half dozen variations and ask people which ones they find easier or more difficult to read. On the web, A/B testing is a standard way to test the effectiveness of an email campaign or a landing page.
Probably the only rule in all of this is: When you're testing a design, don't ask people what they "like" unless it's a purely personal item, like a birthday card or a wedding invitation. Asking what people like in a design focuses attention on the piece itself, not on what it's supposed to do, and you won't get the information you need.