Canoeing with Stephane (Sentiment Analysis, Anyone? (Part 2))

The iMedia Brand Summit has kept me a little busy, and I do keep my promises.

One of the folks I asked about Sentiment Analysis prior to writing Sentiment Analysis, Anyone? (Part 1) was Stephane Hamel. I asked Stephane for a site I could analyze without my knowing anything about their strategy, demographics and such. Stephane suggested canoe.ca since it’s a well known Canadian site that receives lots of traffic and has lots of diversified content.

Canoe French homepage

The Canoe.ca site has an English and a French version so we analyzed the homepages of both versions to demonstrate the differences in cultural cuing. This image is the Canoe French homepage. Below is the English homepage. The information I’m sharing comes out of our tools, specifically the one I described in Sentiment Analysis, Anyone? (Part 1).

Canoe English homepage

This image is the Canoe English homepage. I’ll share at this point that the tool I’m using reads whatever digital information you give it exactly like a human of the intended culture would read it, provide it material in French and it thinks in French, provide it material in Gaelic and it thinks in Gaelic (we get a lot of calls for that, you betcha. The first language our technology understood was Gaelic because if you can do Gaelic you can do anything. Now we’re teaching it Etruscan because you never know when you might want to sell sandals to a dead gladiator). What makes the tool different from the standard human is its ability to report on what will or would happen in the reader’s mind at the non-conscious and conscious levels. Most people don’t have that kind of training, our technology (Evolution Technology or “ET”) does.

Age Appeal

Both homepages are designed for (not necessarily intended for. We’re not talking about who the desired audience is, we’re talking about who this material is going to work best with) relatively tight demographics. The French homepage will appeal to about 71% of the 25-34yo native French speakers who see it, the English homepage will appeal to about 60% of the 35-44yo native English speakers who see it.

<ET Tool Training Alert>
When I originally presented this analysis to Stephane for comment I thought that a possible reason for the different age appeal targeting was that the canoe.ca site was a Quebec specific site, hence English might be a second language — meaning learned via education or life experience — for Canoe visitors (ET will interpret higher levels of education and life experience as “more mature” hence add a few years to its age appeal estimates).

Stephane explained that canoe.ca was created in Toronto then moved into Quebec, and that the English site is still done in Toronto and the French site in Quebec.

In any case, what’s most interesting is the relative spikyness of the Appeal charts. This material — regardless of the intended audience or its origins– is designed to best appeal to a limited age demographic.

<Stephaneism>
Stephane noted:

Another thing… your classifications aren’t equal… why 15-19 (5 years), 35-44 (10 years), 55-59 (5 years)… Does each of the graph age ranges have the same “population size”?
The age groupings are based on neurology more than much else. The five year groups occur when the brain starts to change, the ten year groups are when the brain is relatively stable neurologically.
Usually, I think each segment should be the same range (number of years). If population is different sizes for different ranges it usually mean the number of classes should be reviewed. Am I wrong?

Excellent catch. The age breakdowns are based more on the most recent and most well documented neurology studies than anything else. As such, they can fluctuate from time to time. ET’s basis for understanding and decision making is neuroscientific, not marketing demographics per se. Originally we tailored the age breakdowns to match the US Census bureau’s breakdown and do our best to match those the best we can.

That offered, if you can define the age breakdowns of greatest interest to you (maybe 15-24, 25-39, 40-54, 55-74, … work best for you) we can tell ET and have the results appropriate to your needs.

</Stephaneism>

</ET Tool Training Alert>

Clarity/Understandability

Readers of Sentiment Analysis, Anyone? (Part 1) or Websites: You’ve Only Got 3 Seconds will remember that there are three “age” levels designers really need to be concerned with; Appeal, Clarity and Actionability. The brain-mind system doesn’t “think” in terms of a chronologic age, it “thinks” using one subsystem to determine “Is this going to be important?” (that’s Appeal), another subsystem to determine “Do I understand why this is important?” (that’s Clarity, Cognition, Understandability, call it what you will, god knows we have) and yet another subsystem to determine “Shall I do something about this?” (that’s Actionability).

The chart above shows that both English and French homepages will be best understood by a broad demographic, yes (the curve doesn’t spike), as well as a large population (its position on the chart).

<ET Tool Training Alert>
There is a possible problem when the Appeal and Clarity charts are taken together. The ideal is that Clarity peak at an age demographic just shy of the Appeal peak. This is necessary because humans, once you’ve got their attention, want to quickly determine if something is important or not. This desire to quickly understand something’s importance means less neural activity is required and ET reads that as a slight drop in neurologic age requirements.

However, the Clarity here is above the Appeal of both English and French audiences, meaning both audiences will need to work (as in “think about”) what’s on each page in order to understand its importance to them. If these pages truly are designed for the Appeal spikes, then they will not be easily understood by those age groups, hence Actionability (click through, conversion, whatever) will be lower than it could be.

On the other hand, if the target audience is 35-59yo, this Clarity is fine. Now the problem is that the age group will not find the homepages appealing enough to devote time or energy to them (except possibly some percentage of native English speakers), meaning “your conversions/clickthroughs/… would be higher with a judicious redesign”.
</ET Tool Training Alert>

Actionability (conversions, clickthroughs, …)

Both sites are designed to be actionable by 35-44yo. This is great for the French site (and assuming it is correctly designed for its intended audience) and not so good for the English site. Actionability needs to be a tad more than the Appeal because action requires effort and ET reports this as an increase in neurologic activity, hence a shift to a more mature age group.

<ET Tool Training Alert>
The good news for the French site is that the Actionability spike is pretty much as the same height as the Appeal spike and it’s in the correct demographic. This means every native French speaker who comes to the French homepage will act on it.

Unfortunately, the Clarity value is way off from where it should be. Native French speaking visitors may find the site appealing and be able to act upon it but they will not understand what it is they should do, hence numbers could be higher with some redesigns.

The English Actionability is acceptable and is also quite the spike. It almost matches the Appeal spike, but the page also suffers from the Clarity issue.

</ET Tool Training Alert>

Gender

Both sites favor a male audience design wise and in roughly equal measure.

Rich Personae, {C,B/e,M} Matrix

Often this is where real cultural design differences make themselves known. The English site is designed for an A9 Rich Persona (I’ve written about Rich Persona on this blog and in iMediaConnection), the French site for a V16 Rich Persona.

The A9 Rich Persona has the following attributes when it encounters web based information:

  • These people focus on the negative, they make decisions based on what might go wrong
  • They are motivated to take action when things are phrased in the negative
  • They often need to confirm their beliefs with visual information
  • They’re motivated by avoiding trouble and are strongly influenced by the possibilities of difficulties down the road

The V16 attributes are:

  • These people need to have information presented to them in pictures, charts or graphs
  • They finalize their decisions by using internal dialog
  • They need information framed in a positive manner before they can accept it
  • They have no sense of time or process

So we immediately see that the French homepage is designed for happier people than the English page.

<ET Tool Training Alert>
The fact that the two sites target completely different personality types can be a plus or a minus based on how much of the Canoe visitor populations match these psychological profiles. What is most important is that what is essentially the same design will target very different psychologies based on the native language of the visitor.

Which personality profile is better? Couldn’t tell you without knowing more about the goals for the site.

</ET Tool Training Alert>

10 Must Messages

10%20must%20messages%200906081039-small.jpg

The basis for communication and relationship are what NextStage calls “The 10 Must Messages”, meaning unless your site is communicating this messages well your site won’t work at all.

<Aside>
Interestingly enough, during the iMedia Brand Summit Master’s Class I taught earlier this week I asked all the attendees what the basic function of a website was. There were lots of answers and none of them were the most important one; to establish a relationship between the visitor and the brand. Regardless of intent, a relationship is being established and the success of that relationship is going to be based on how well the site communicates these messages to the visitors.
</Aside>

What we see here is something I mentioned in Sentiment Analysis, Anyone? (Part 1), that Canadian based companies tend to shout “We’re a Leader”. The fact that the two lines have roughly the same shape is to be expected (my guess is the same design group handled both homepages or a single template was used for both). Again we see some cultural based differences in the strength of the messaging.

<ET Tool Training Alert>
Take each line separately and the values are fair, there’s not a lot of shouting. What is a problem for both sites is the “This Is Important” message’s relative weakness. It is so low compared to most other messages on either site that visitors will feel no sense of urgency, no impulse to act, and in any case nowhere near as strong as it could be. The ideal would be for the “This Is Important” message and the “This Is Important To You” message to be high with the latter just enough higher to have visitors non-consciously recognize the difference.

I tend to liken the difference between these two messages to hearing the newscaster tell you about some news story then call in their talking-head to explain specifically why this news story is important to the viewer. Another way of thinking about their difference is the recognition that something may be important but not relevant to the individual versus important and relevant.

In any case, you can’t convince people that something is both important and relevant unless you first convince them that it’s important, period.

</ET Tool Training Alert>

Suggestions

That brings us to the last thing ET will report on, what to do to change the design for the target audience. I don’t know who the target is so any suggestions would be irrelevant, me thinks.

<Stephaneism>
After reading this analysis, Stephane commented:

I think what’s also interesting is ET gives you the data and the charts, but you still have to know that “Actionability needs to be a tad more than the Appeal because action requires effort”. The next stage of ET (no pun intended!) could involve bringing this “higher intelligence” (your intelligence!) to a rule engine that would gradually integrate this additional knowledge.
Let me take an example… web analytics tools today collect, analyze and provide the data, but they don’t provide any insight. Yet, some rules are readily applicable if we see high traffic from a specific campaign but a lower conversion rate than average: incoming traffic is less qualified, the campaign might need to be realigned. This intelligence could be integrated directly into the tool to raise “alarms” when things like this happen. The system would need to be trained and the architecture should allow to include new rules easily.

This is an excellent thought and yes, we’ve got it covered. People who’ve heard or seen my presentations know that one of ET’s differentiators is its ability to make suggestions. The tool that produces these reports — the one that doesn’t need a tag on a client’s site to generate actionable results — provides suggestions that incorporate “my intelligence” and additional knowledge (the system borrows heavily from knowledge management systems I worked on several years back) into its analysis. If I understand the rules system you’re describing, it’s already in there.

Anyway, we’re currently in the process of looking for alpha clients to help us integrate those rule engines into the product that does these analyses. [[(Already done and in NextStage OnSite, NextStage Experience Optimizer, NextStage Immediate Sentiment and NextStage Veritas Gauge)]]

</Stephaneism>

And there you go, Stephane. Hope it’s useful.


Posted in , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Improve Design to Increase Usability and Brand Loyalty

Note: This originally appeared as a NextStage Evolution Case Study

The Client

A F500 company engaged Columbia, South Carolina based MarketSearch Corporation to conduct a usability analysis of its website. The study focused on employment and investor relations and compared the company’s site to two other corporate sites.

The Problem

The standard usability paradigm, while well known and understood, doesn’t necessarily provide the most accurate results. According to Eric Drouart, Former VP, International Operations, Bristol-Myers Squibb:

Marketing research methodologies that rely on questionnaires and standard surveys are inherently loaded with biases and errors related to the sampling frame, the survey instruments, the interviewers and the fact that the respondents know that they are being evaluated. NextStage is truly a non-biasing research tool with a lot of validity and reliability because it is based on non-conscious responses to information. This methodology offers a lot of advantages over traditional methods to evaluate the appeal and the benefits of a web site.

Charles Wentworth, who directed the study for MarketSearch, appreciated the possibility of unintentional biasing and wanted a tool which would allow him to perform a separate analysis of the study participants. Specifically, Wentworth wanted a tool which would allow him to focus a separate non-intrusive “lens” on the participants while they were engaged in the study to determine if the knowledge of their participation was affecting the results.

Wentworth engaged NextStage to evaluate media content with its Evolution TechnologyTM and provide real-time usability measurements. By using NextStage’s Evolution TechnologyTM [[(what’s now broken out into several tools; Sentiment Analysis, OnSite, Age Persuader, Gender Persuader Tools and NextStage’s BlueSky Meter)]] side-by-side with traditional analysis methods, Wentworth was able to determine correlations between the two methodologies as well as perform exhaustive research in a minimal time frame. These tools allowed Wentworth to determine the client’s branding effectiveness vs branding effectiveness of other corporate sites.

The Solution

Wentworth developed the research paradigm and solicited participants. Preparing the materials for NextStage’s analysis took no more than an hour’s time and involved nothing more than providing NextStage with some webpage pointers and editing a few web files.

Study participants were questioned and observed by Wentworth as well as by NextStage’s tools, the latter providing an impartial, electronic observer to both the participants’ interactions with the websites and with Wentworth as an inquisitor.

The Result

Wentworth comments:

NextStage’s Evolution Technology allowed us to assess a host of usage issues that just can’t be measured using conventional testing approaches. It gave us an opportunity to go a lot deeper, and the result was something much more valuable to our client than a typical study would have been. The correlation between what we saw during the interviews and what NextStage’s technology picked up behind the scenes was startling, and we know much more, now, about website visitor behaviors, demographics and psychological factors than we would have.

We’re eager to use it again in future studies and see just how far we can push it. We see a lot more potential than what we originally expected from it.

NextStage’s Evolution TechnologyTM suggested ways to increase brand awareness which resulted in a measurable 35% increase in the client’s brand effectiveness. NextStage was able to make suggestions which increased the study group participant’s subjective ease of navigation (“usability”) greatly, which also contributed to brand effectiveness.


Posted in , , , , , , , , , , , ,