Introduction: The Product Experience
The user sits at the center of the product experience, and their sentiment towards the product depends on a variety of factors. Visual (how does it look), tactile (how does it feel), and aural (how does it sound). It might be easy to understand a sleek visual aesthetic, but how would you pursue a sleek sound? Or how can brand iconography extend to create recognizable sounds that stand out like Netflix’s “tudum”? Ultimately, products don’t exist in a vacuum, and the context in which the product is used can change the perception of its sound.
The best feedback for how a product should sound comes directly from the users, through listening panels. Quantitative understanding then comes from analyzing their responses and combining them with rigorous noise audits to understand the sources of noise and how to change them. At Acentech, we consider acoustics a blend of art and science, and it is impossible to rely on either domain alone to craft successful product sound designs.
Investigation: Noise Audits
A single product has many components that contribute to its overall noise character. These sources might include noise radiated directly from a component under normal operation, such as the whine of a motor or the tonal noise associated with a fan’s blade passage frequency. Other sounds are only produced when the components are interconnected, like structure-borne noise from the same motor connected to the body of the product, or noise from gear teeth meshing together. Other interactions may be dependent on non-direct connections, such as airflow noise.
As noise control engineers, we conduct “noise audits” to break down the sound into as many principal components as possible. We measure individual sources in a reverberant test room, and when the source object can’t be isolated (by, for example, operating it alone under representative loading conditions, or by signal processing means), we use a “, where other sources are attenuated with passive enclosures, wrappings, or silencers. Sound intensity probes or acoustic cameras can also be used sometimes to get more detailed estimates of source contributions.
Once the sources have been identified and quantified, they can be rank-ordered to prioritize the development of noise reduction for the loudest components. This ensures that time isn’t spent attempting to treat components that aren’t significant contributors. From there, a new composite sound can be rendered, mixing various components at different levels to achieve one possible version of a desired product sound. One option might be a balanced design, where each major source has roughly the same contribution, or an unbalanced design, if one component is preferred to be the principal sound for the product. Making these adjustments purely in terms of the overall A-weighted sound level can be an effective way to reduce the noise, but they don’t necessarily represent the user’s perception. Here, we turn to the users to more effectively home in on a design.
Understanding User Experience: Sound Quality Jury Studies
The sound a product produces can contribute to a person’s overall perception of the product or brand. Users may judge the product based on the overall impression conveyed by the sound or the acceptability of the sound in the context of its use (including whether the person is a user of the product or a bystander). Companies are often faced with negative reactions to the sound of their products and may be uncertain as to how to improve the sound quality to increase the general acceptance of their products. These issues, coupled with the need for manufacturers to develop high-end and international markets, have made sound quality (SQ) an important product attribute.
Acentech designs and carries out unique and customized forms of SQ jury studies that are used to evaluate and document the subjective impressions of a range of “what-if” sounds produced by a product. We then provide guidance on how to modify the product to improve its sound by a targeted amount, in agreement with the jury study outcome.
Many dimensions affect the acceptability of a product’s sound, or even the perceived build quality of the product, beyond the raw decibels. Some of these include:
- Strength/magnitude (dBA, loudness, speech interference)
- Annoyance value (noisiness, roughness, sharpness, Perceived Noise (PN) dB, tonal prominence – bothersome aspects of the sound)
- Amenity value (regularity, harmonicity, pleasing aspects)
- Information content (identification, performance, condition of the product, appropriateness of the sound relative to context)
Figure 1: Example response surface profile generated for the perceived build quality of an air tool (where 0 is “very poor quality” and 100 is “excellent quality”)
Some of these can be obtained through measurement alone, while others require input from the users. User input can be especially valuable as users consider context in terms of expectations for the sound of a particular type of product based on their experience, usage environment, and so forth. Ultimately, sound quality is a response by people, not a meter. As sound designers, we need to gather a mixture of sound-related and non-sound-related information from the users. One method to obtain this information involves the use of focus groups (to provide an initial assessment of the relative importance of sound compared to other product attributes) and jury studies, made up of a representative cross-section of actual consumers for the targeted market. The users may not be able to express in words what they want a certain product to sound like, but when presented with the sounds of different versions of a particular product in a jury study, they can rate or pick and choose which versions they prefer.
As product sound designers and mechanical engineers, we can modify the characteristics of various components of a recorded product sound to simulate a range of possible sound profiles, which are constrained by modifications that are actually feasible to implement. By analyzing all of the user feedback across this matrix of potentials, we can generate response surface profiles like that shown below in Figure 1 to indicate how the design might develop to create a more desirable sound. Plots like this can reveal many possible paths for improving the sound with relatively equal levels of satisfaction (i.e., going from the current design to anywhere along one of the contours that have the same increased value for a particular subjective attribute), keeping in mind that some of these designs may be much easier to achieve than others. It is even possible to glean further information from the users, such as how much more they might pay for a product with better sound quality.
Sound Quality Jury Study Design: Example Process
There are several factors that go into designing a jury test for sound, from choosing the basic type of study (magnitude estimation, semantic differentials, paired comparisons, etc.) to deciding on values for a number of parameters that will affect the logistics, type of information obtained, and statistical confidence associated with that information. All of these require careful consideration well before the listening panel convenes.
To illustrate this process, we can review the elements needed for an example magnitude estimation type of study, in which jurors were asked to provide their numerical ratings on a set of “virtual” vacuum cleaners that they listened to. For this particular study, the client wanted us to determine ways in which their vacuum could be changed to make it sound more “acceptable” while at the same time not decreasing the perception of its power, as conveyed by its sound.
The basic concept for this type of study borrows from the food/flavor and perfume industries, where a common goal is to determine an optimal mix of ingredients that will increase the user experience. In our case, the “ingredients” are the sounds produced by the different components making up the product, and the “enjoyment” is reflected by ratings on subjective attributes such as “acceptability” and “power” or “effectiveness”, as conveyed by the sound of the product. For vacuum cleaners, we can think of there being, as denoted in Figure 2, each producing its own unique sound:
1. The Suction Fan: High Pitch Tone
2. The Motor: Mid-Frequency Tones
3. Rotating Brush: Low Frequency Tones and Possible Rattle
4. Nozzle Airflow: Random Noise
Figure 2: Main components responsible for producing sounds in canister and upright vacuum cleaners
Once we obtain the sound of each of these four different components, we can then construct the sounds of several “what-if” vacuums by modifying these component sounds in different ways and then mixing them back together to form these virtual vacuum cleaner sounds. One modification is to simply vary the overall sound level of each component, (subject to constraints on what is actually likely to be feasible from a product engineering standpoint). Other types of modifications (such as tonal harmonic content, modulation, frequency-dependent attenuation, etc.) can also be considered, as long as they are quantifiable in some fashion. In the end, the goal of this process is to determine a regression relationship between quantifiable changes made to these component sounds and any observed changes in attribute ratings as provided by the jury.
Exhibit 1 | Interactive Listening Module
Listen to the separate vacuum sounds here:
Motor Sound (w/. cooling fan) (M)
Airflow Sound (A)
Agitator (Beater) Sound (B)
Suction Fan Blade Passage Tone (S)
Below, there are six different samples, each having the different parts of the vacuum sounds edited louder, quieter, or no change made. Such changes were made and tracked by changing the loudness by 4 decibels at a time, represented as one “step”. (1 Step = 4 Decibels)
Listen to the edited versions below, then vote on which you prefer.
Variation A
Variation B
Variation C
Variation D
Variation E
Variation F
What were the differences?
What changed?
Variables M, B, and S were increased by 1 step and variable A was decreased by 1 step.
Original:
Variation A
What changed?
Variables A and S were increased by 1 step and variables M and B were decreased by 1 step.
Original:
Variation B:
What changed?
Variable M was decreased by 2 steps.
Original:
Variation C:
What changed?
No variables were changed.
Original:
Variation D:
What changed?
Variable A was increased by 2 steps.
Original:
Variation E:
What changed?
Variable M was increased by 1 step and variables A,B, and S were decreased by one step.
Original:
Variation F:
Generally, the range of varied component sounds and the number of combinations required to adequately cover the design space are determined by a statistical Design-of-Experiments (DOE) methodology. We often use a Central Composite design for this, but other DOE types, such as Simplex and Box-Behnken, are also possible. The DOE type, along with the number of component sounds, the number of operating modes, the number of subjective attributes to rate, the desired confidence level, and whether to perform a 1st or 2nd order regression (or include interaction terms), all influence the number of jurors needed and, along with the time duration of the sound samples, determine the overall length of the jury sessions. Sound samples should be kept fairly short, if possible (say, 5 to 10 seconds), as our “sonic memory” is generally pretty limited.
For the vacuum cleaner example, we needed to obtain representative sounds of each of the four components depicted in Figure 2. This was accomplished by a combination of signal processing and recording sounds while operating individual components by themselves., We built a rig so that the motor could be properly loaded when run by itself (since the sound of a motor under load is different from that under no load), and utilized custom-built acoustical enclosures to isolate the airflow sound from the rest of the vacuum cleaner sound. The top part of Exhibit 1 contains the final sounds obtained for the four components of the vacuum.
The central composite statistical design utilized for this particular study governed how these component sounds should vary and how many different combinations should be presented to the jury. The contains an example sequence of various “what-if” vacuum cleaner sounds, all obtained by mixing together different level-varied versions of the component sounds, as shown. After a training session, the jury listened to each “virtual” vacuum cleaner sound through a loudspeaker* and provided numerical ratings of the sound on two attributes: “Acceptability” and “Perceived Power”. Several sequences of sounds like those listed in Exhibit 1 were presented, covering a range of possible vacuum cleaner sounds.
*(for typical products like an appliance, it is generally easier to preserve the spatial sense of the product by using loudspeakers to play back the sounds rather than headphones
Jury Study Analysis: Example Process
The attribute ratings from all 25 jurors in our example vacuum cleaner study were first reviewed to determine if there were any jurors who provided no ratings or gave unlikely ratings (such as the same number for all sounds). The ratings were then used as inputs to a response surface analysis configured to utilize multiple linear regressions to build a quadratic model of the relationships between the observed Acceptability and Power ratings and the component sound level changes. Basically, this process tries to fit the entire set of individual juror ratings (around 2,000 observations in total) to a response surface model with all the linear, squared, and two-way interaction terms included. After eliminating a first round of observations with large, standardized residuals (around 4% of the total number of observations) and eliminating statistically insignificant coefficient terms, the analysis was re-run, resulting in response surface models for each attribute.
The regression equations resulting from this reduced-order regression analysis (i.e., the “models”) are summarized below in Figure 3. These models relate the jury-rated values for “Acceptability” and “Perceived Power” to changes in the component sounds from their baseline (i.e., as-recorded) conditions.
Figure 3: Reduced-order regression equations obtained for “Acceptability” and “Perceived Power” of an upright vacuum
The models can then be used to predict the effect on Acceptability and Power based on changes in the various sound-producing components in the vacuum cleaner. In these equations, the current baseline average values for Acceptability and Power are given by the constant term that appears right after the equals sign. Figure 3 also includes an example contour plot similar to that shown in Figure 1( in this case derived from the equation for vacuum noise Acceptability as the motor and airflow sound levels are changed while those for the suction fan tone and the brush are kept constant, i.e. at their original values.
Associated with each of the equations shown in Figure 3 are various statistical indicators arising out of the regression analysis, some of which can be used to help assess the reliability and accuracy of the models. One of these indicators is the regression coefficient, R2, which is the percentage of the observed variability that can be explained by the given model/equation. We have typically seen R2 values in the 40% to 60% range in these types of SQ jury studies that we have conducted on consumer products and other devices such as medical devices, delivery drones and farm equipment. Variability in juror judgments can be fairly large. That does not mean that predictions to changes in component sound levels are wrong, but simply that the design change will not necessarily be universally recognized as beneficial. That is simply a fact of how people perceive sounds. The regression equations themselves are but one tool that can be used to guide subsequent engineering efforts in the pursuit of increasing the end-user’s positive perceptions of the product, as conveyed by its sound.
Using Jury Study Results to Increase Sound Quality: Example Process
In general, regression models such as those shown in Figures 2 and 3 can be used to mock up a few plausible “candidate” or target designs that should improve attribute ratings by similar amounts. These candidate designs are then presented to a jury to determine whether there is a real preference for one design over another, and if so to what degree. The main result from such a “verification study”, usually carried out via forced-choice paired comparisons, will be a preference ranking or “score” of the sounds of the different candidate designs (along with, optionally, the baseline sound of the unaltered design). This information can then be used to assess how close or far apart these designs are to each other in terms of preference based on sound, and to help select a final design for implementation and testing.
Getting back to the vacuum cleaner example, the results shown in Figure 3 indicate that the motor and airflow sounds contribute the strongest to Acceptability and Perceived Power. They also reveal that a higher level of beater brush noise increases Power more than it decreases Acceptability, suggesting a possible route to somewhat increase the perception of power without decreasing Acceptability too much. These results also indicate that the suction fan tone and airflow noise both have a negative correlation with Acceptability, and little effect otherwise, which says that these components can be usefully reduced to improve acceptability without adversely affecting the perception of power.
Figure 4: Wire mesh model of vacuum cleaner motor showing vibration response at 500 Hz
We explored one way to increase Acceptability while minimizing a decrease in Power by redesigning the vacuum cleaner motor for reduced noise. In a subsequent motor study that we conducted, we found that the radiated motor sound was due mainly to rotating imbalance and electromagnetic interactions between the rotor and stator in 300-2,000 Hz range, and that structural resonances in this frequency range, excited by electromagnetic forces paired between stator and rotor, dominated the vibration response in this range. (Such vibration often leads to the generation of radiated noise).
Figure 4 shows an example of the resonant vibration response of the motor for a mode at 500 Hz. Equivalent imbalance and electromagnetic forces were then determined experimentally by inferring them from operating vibrations and measured transfer functions, which suggested a redesign that incorporated structural stiffening in a manner consistent with the motor mold design. This stiffened design reduced the broadband motor vibration levels by 5 to 10 dB, resulting in lower noise levels for the motor.
Beyond Jury Studies: Correlating Sound Quality Metrics to Jury Results
It can sometimes be beneficial if physical metrics calculable from the measured sound of a product can be correlated to existing sound quality ratings that have already been obtained for that particular type of product. To do this, a modified form of Principal Components Analysis (PCA) or Partial Least Squares (PLS) can sometimes be harnessed to reduce the many existing sound quality metrics that can be calculated to just a few custom metrics. Usually, these turn out to be weighted combinations of the most “important” metrics as determined by PCA or PLS, which can then be used to relate physical measures of the sound from existing and future products to sound quality as determined by existing jury ratings. This can make it convenient to use the recorded/measured sounds of various potential prototype products to predict subjective reactions to their sounds without necessarily having to conduct further jury studies on different versions.
Design Cycles: Where Sound Quality Fits
Without intentional steps to shape a product’s sound quality, consumer products tend to become louder due to market pressures to use lightweight components to reduce costs or have faster-rotating parts to achieve higher performance. Acentech’s Product Sound team works directly with manufacturers’ representatives from product planning, marketing, engineering design, heat transfer, and other specialty areas to balance these design goals with the implications on sound quality.
Finding the time and budget to engineer an improved product sound is challenging. If you start too early in the design, you are limited in the ability to predict realistic sound characteristics, but too late and you may not be able to change critical components. By incorporating sound quality analysis with noise and vibration control at the right time, you are ensuring the most cost-effective means of product sound design. Once you have conducted one noise audit on a product line, you can use those results to inform future model versions earlier in their design. With these thoughtful design strategies, manufacturers can bring products to market that sound better, thereby enhancing the consumer’s overall product experience.