Reading Virtual Minds Volume I: Science and History, 4th edition

It’s with great pleasure and a little pride that we announce Reading Virtual Minds Volume I: Science and History, 4th EDITION.

Reading Virtual Minds V1: Science and History, 4th edThat “4th EDITION” part is important. We know lots of people are waiting for Reading Virtual Minds Volume II: Experience and Expectation and it’s next in the queue.

But until then…

Reading Virtual Minds Volume I: Science and History, 4th EDITION is about 100 pages longer than the previous editions and about 10$US cheaper. Why? Because Reading Virtual Minds Volume II: Experience and Expectation is next in the queue.

Some Notes About This Book

I’m actually writing Reading Virtual Minds Volume II: Experience and Expectation right now. In the process of doing that, we realized we needed to add an index to this book. We also wanted to make a full color ebook version available to NextStage Members (it’s a download on the Member welcome page. And if you’re not already a member, what are you waiting for?)

In the process of making a full color version, we realized we’d misplaced some of the original slides and, of course, the charting software had changed since we originally published this volume (same information, different charting system). Also Susan and Jennifer “The Editress” Day wanted the images standardized as much as possible.

We included an Appendix B – Proofs (starting on page 187) for the curious and updated Appendix C – Further Readings (starting on page 236). We migrated a blog used for reference purposes so there may be more or less reference sources and modified some sections with more recent information.

So this edition has a few more pages and a few different pages. It may have an extra quote or two floating around.

You also need to know that Reading Virtual Minds Volume I: Science and History is a “Let’s explore the possibilities” book, not a “How to do it” book. As such, it deals with how NextStage did it (not to mention things that happened along the way). It does not explain how you can do it. This book’s purpose is to open a new territory to you and give you some basic tools for exploration.

There are no magic bullets, quick fixes, simple demonstrations, et cetera, that will turn you into jedis, gurus, kings, queens, samurai, rock stars, mavens, heroes, thought leaders, so on and so forth.

How to Do It starts with Volume II: Experience and Expectation and continues through future volumes in this series. We’ve included a Volume II: Experience and Expectation preview with a How to Do It example on page 302 so you can take a peek if that’s your interest.

That noted, I’m quite sure that you won’t get the full benefit of future volumes without reading this one because unless you’ve read this one you won’t understand the territory you’re exploring in those future volumes.

Reading Virtual Minds V1: Science and History, 4th edThat’s Reading Virtual Minds Volume I: Science and History, 4th EDITION. It’s so good and so good for you! Buy a copy or two today!

Posted in , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Have you met Eris? You might have known her as Xena

A) I’m not talking about the Warrior Princess getting a makeover and B) just something I found interesting, another swipe at the mythologies I grew up with.

Quick like a bunny, How many planets are there? Eight? Nine? Ten? Bigger than a breadbox?

I think and am not sure that there are eight planets. Pluto, an old and cold (but not cold-hearted) friend I grew up with has officially been dwarf-planeted. No longer a planet but neither an also-ran, Pluto is too small to be an official “planet”. Once again, humanic egotism decides the rules of the cosmos.

Eris (also known as Xena) and her moon, Dysnomia (also known as Dysnomia)

And not only is Pluto no longer a planet, Pluto is not even the largest dwarf-planet. That honor goes to Eris, a dwarf-planet known as “Xena” to her friends, shown here with her moon, Dysnomia, also known as Dysnomia to her friends. Eris-Xena is 27% more massive than Pluto. Not sure where to go with that one. Does he need to beef up or does she need to go on a diet?

And how does this tie into the myths of childhood?

Jupiter, comin' at cha

Does anybody remember when Jupiter’s Great Red Spot of Jupiter was whacked by Comet Shoemaker-Levy 9? I remember an NPR talk show (I think it was years after the fact) dealing with it and possible Armageddon (of course. I wonder if the Jovians were concerned?). I remember my earliest science readings (probably in Reader’s Digest) about the Great Red Spot; it was bigger than earth, you know.

Comet Shoemaker-Levy 9 whacks Jupiter

Forget the ripples this created in the Jovian atmosphere. Forget that we (Earth) could be next and here was an example of just how devastating such impacts could be.

Such things are abstract in the extreme to most people walking the planet. The human mind really can’t understand these things except at a superficial level. We’re still pretty much stuck with a “Find food. Stay warm. Reproduce” wiring.

But in that wiring are things we decide have permanence. One of the ways this concept of permanence makes itself known is in the catchphrases “It’s not rocket science” and “It’s not brain surgery”. We as a culture have decided these are difficult tasks and have elevated them to “permanent-difficult” status. These elevations can be extremely personal and private and can be thought of as personal myths.

For me, one of the things I elevated to permanent-trust status was that the planets were, well, the planets. One reason lots of people had a challenge with Pluto being demoted to dwarf-planet status is because that permanent-trust elevation was being violated.

So when The Great Red Spot got whacked? Ohh, c’est terrible! Thank goodness that when I wikipeded The Great Red Spot I learned it was still there. I think. This was Wikipedia, after all.

Links for this post:

Posted in , , , ,

The Complete “The Curse of Knowledge” Arc

Note: a short, two post arc, presented here in all its glory and for your enjoyment -C&E

The Curse of Knowledge, Part 1

I wrote a while back about working with Multitasking, anyone? a company that had a statistician — not! This company hired NextStage to help their workers become more productive and we’ve had someone assigned there for several months now, primarily data gathering and modeling. The modeling method we’re using is called process modeling, something very well recognized in cultural and social anthropology and apparently not recognized at all in HR or B2E systems.

People who’ve seen a National Geographics, Nature, Nova or similar show (or who remember when Disney Studios did nature documentaries. I loved those and probably still would) highlighting a tribal culture have seen process modeling without recognizing it. Anyone who’s had “on the job” training has taken part in it. People with children or who work with children practice it without knowing what it’s called. Anyone who’s taken part in an Apprentice-Journeyman-Craftsman-Master career track has done it and probably not known it.

The mother who cooks while her daughter or son is first watching then helping then experimentally cooking preparing the main meal while mother looks on is learning via process modeling. The tribal elder who makes a flint tool while the youngster first watches then helps then tries a few knaps then proudly makes a high-quality spear tip is learning via process modeling.

Process modeling is what occurs when we demonstrate success without any requirement that the other be successful, only that they attempt success when they’re comfortable doing so. It is one of the most powerful and primitive teaching methods known and not often practiced formally as such anymore. It also works best with sensory skills rather than cognitive based skills.

So, anyway, we first had one person demonstrating success via process modeling for several months now and recently added a second one. Their individual success rates are both close to 90% and this is phenomenal considering most and not all others in the company are lucky to reach 10-15%. It’s taken a while and in the past two months other workers and management (the ones who don’t know the NextStagers are in their midst) have come to ask how the NextStagers are being so successful.

A five minute “training” increased one fellow’s productivity 150% in four hours. He was gleeful with success.

click on image to download a larger one

How successful was this success en large? Over a five month period ten workers selected at random showed the following increases in productivity. Those right most columns are labeled P1 and P2. They are the process modelers. The other columns, A through J, are the ten workers. Process modeling takes time and that’s what gives it much of its power. In this case, five months and at the end of those five months all had richly improved and four of the ten were working at the modelers’ levels. Those four — once the NextStagers are through and move on — will become the modelers for the next “generation” of employees. This is how learning is passed on generationally in all cultures. Without such learnings cultures and civilizations die.

I bring this up and demonstrate this increase in productivity because it goes back to the statistician who isn’t what I’d consider a statistician at all.

The Curse of Knowledge, Part 2

The truth is, process modeling is always successful when certain base line conditions are met. But how to explain this to a statistician, especially explaining to the statistician that the numbers he was looking at weren’t where he should have been looking.

A real challenge in any educational environment, especially ones in which a given methodology has acquired protected status, is convincing stakeholders that the protected methodology is invalid. This problem is compounded by surrounding that educational environment with a business paradigm. It’s kind of the enigma wrapped in an illusion. Put a bow of misunderstanding your metrics on top and you have The Curse of Knowledge; someone knows just enough to convince others to look in the incorrect place for something that doesn’t exist there, then uses whatever they find to claim understanding.

The Curse of Knowledge. It’s not just for breakfast anymore!

Posted in , , , , , , , , , , , , , , , , ,

I did a little research…

I’ve mentioned in several places how much time I spend in research. There’s an interesting (to me) anecdote tied to that which I’d like to share.

I was in a meeting with a US$5MM company describing what NextStage does. Eventually the questions got around to “How did you figure this out?”

My reply (same now as it was then) was, “I did a little research.” It is, I believe, a truthful statement.

Many of the people in the room, however, rolled their eyes. Some pushed themselves away from the table we were sitting around. Most of them looked at my host as if to say, “Why are you wasting my time with this yutz?”

My host — a woman I consider one of my mentors (Howdy, Laureen!) — moved her hand slightly, a signal to everyone in attendance to wait a second before leaving the room. She asked,

Joseph, can you describe your research, please?

Me, nufratedes (Italian for “Clueless in Seattle”), said matter of factly,

Oh, I was listening to a conversation during lunch as MSU in ’87. That got me started, I published my thesis in ’91 and then…

It was 2003 in which this conversation occurred, you understand.

One of the people at the table replaced his jaw because it had dropped to the floor. “Fifteen-sixteen years and you call that ‘a little research’?”

“Yes, why?”

“To me ‘a little research’ means you looked up something on the web and maybe read one or two papers on the subject.”

This time my jaw dropped. “That would be foolish and a waste of time.”

“What do you call ‘a lot of research’?”

“Oh, I don’t know…a longitudinal study, I guess…sixty-five, seventy years, perhaps.”

Did I ever mention that NextStage Evolution is a research company. NextStage Global [[(Susan closed it down shortly after she took over, hence it is no more)]] productizes and markets what comes from our research [[(Most of this is just available to our members and clients now. And you should be one if you’re not already!)]].


Posted in , , , , , , , , , , , ,

I’m the Intersection of Four Statements

My guess is that this post is going to get a bit thick for some readers once they get below the fold. We’re going to get into Modality Engineering, something that NextStage’s Language Engines do when they do…well…anything. Our “sentiment analysis” tool uses equations derived from Modality Engineering to make its determinations. We have a whitepaper, A Primer on Modality Engineering, that we’re currently updating and is most likely why this blog post occurred to me. I’ll let you know when the update is available.

Anyway, my hope is that you read this post and enjoy the ride. There’ll be some pretty pictures if nothing else. And no quizzes at the end.

And for those of you who have no appreciation for nice country rides there’s a “What am I suppose to do with this, Joseph?” link after the fold and before we go for the ride.

Which is a pity, you know. We usually stop for ice cream (the good kind made with lots of real cream and natural flavors. The place on the Commons in Barre, MA, is great) along the way.

A long time ago (mid to late 1980s) I studied mathematical linguistics (I still do, as much as one can).

Unfortunately, mathematical linguistics no longer exists as a discipline (as far as I know). The late 1980s was when recognizable computing power moved from computer rooms to desktops and cycle time became cheap. Mathematical linguistics quickly became computational linguistics because it was easier (I’m guessing here) for people to use software than to actually understand the underlying principles of what they were doing.

And the above paragraph is an excellent example of why I’m the intersection of four statements.

I knew a woman who had a PhD in planetary atmospherics. Her specialty was Saturn, I think. I was impressed until I talked with her. I explained how a few things in ET make use of toroidal equations and can be likened to Bernoulli forces. Her face went blank. I excused myself, perhaps I wasn’t using the correct terms.

Oh no, I was the correct terms. She just never bothered with that. She always made sure she worked with someone who knew how to use the computer so she wouldn’t have to learn that stuff.

“You have a PhD in planetary atmospherics and never learned meteorologic equations, atmospheric dynamics, things like that?”

Her excuse was “Well, I never intended to use my degree.”

I also knew a woman at Dartmouth who was granted her PhD in biochemistry because she was nine months pregnant and her husband got a job two states away. Her research wasn’t that great and couldn’t be completed but again, she never intended on using her degree (she told us that) so it was granted to her to make room for someone else. Someone else equally intentioned, no doubt.

There’s a reason for my emPHAsis in the above. They are more examples of why I’m the intersection of four statements.

This week four different individuals made four different statements about me based on their personal experience of me. My mathematical linguistics nerves tingled because I recognized the four statements as mathematically orthogonal and that “I” was the intersection of their four statements.

This means one could mathematically determine how well the five of us would get along together should the situation ever arise. Mathematically, are there certain conditions such that all four statements must be simultaneously true?

Whoa!, don’t you think?

For those who don’t like ice cream, What am I suppose to do with this, Joseph?.

The four statements are:

  1. You are somebody that transmits confidence. And I have to admit that sometimes I’m concerned when I want to convince you of something as I know you’ll know more facts than I will and it won’t matter what we’re talking about.
  2. You have a knack for bringing out people’s lack of self-confidence. People don’t like to be corrected, even in private (I’ve never seen you publicly correct anybody) and when you correct someone (even in private) it’s like being hit by a firehose. “How does he know all this stuff?” People have faith in your being correct and in their being wrong.
  3. Your commitment to the truth outweighs your commitment to people’s feelings — particularly when dealing with idiots or ?ssholes — but it’s a close race.
  4. Everybody I know thinks you’re a great guy. Everybody likes you. But everybody’s intimidated by you, too, because they go around saying they’re experts in something, you say you’re not an expert in anything and you always know more about their field than they do so they end up feeling like imposters.

I was particularly taken by the use of the term “imposter”. I’ve met lots of people whom I recognize as dealing with what’s called “The Imposter Syndrome” — a core belief that they are not able to do what they claim to be able to do, basically not who they say they are or that what they do doesn’t do what it claims to do (I cover this topic in Reading Virtual Minds). I’ll point them out to you the next time we’re at a conference together.

That will get me invited to lots of conferences, don’t you think?

And if you really knew me you’d know I would never do this, that it’s a point I would not cross, something two of the people I quote above understand even if they never thought of it as such.

It’s amusing — I consider myself one of the least confident people I know. I check my data and sources several times before publishing anything, pass things out to first readers (would you like to be one? Let me know) and cherish their comments, most often acting on them, and even then I sit on things until the little genie inside says it’s okay to set some research free. That’s not a sign of confidence to me. Analness, probably, regarding the repeated source checking and data analysis. Thoroughness, I’d like to think. Anybody who’s seen whitepapers I’ve written knows I like to document my sources. One of my editors told me that his staff had a weekly pool; How many links would Joseph have in his next column?

I was also shocked that a fairly high executive of a very large search company didn’t comment negatively during a presentation because, before he raised his hand, I commented that I thought the presentation was great. He told me afterwards that he didn’t want to appear like a fool. “If you thought it was good then it had to be.”

Most people don’t recognize that language and mathematics are both symbolic representations of internal reality. Mathematics (truly) is nothing more than a language itself, merely a specialized language. For that matter, English is a specialized language. Only people who “know” English can understand it or even recognize it when spoken. The same is true for French, Mandarin, Lakotah, … Want to have fun sometime? Listen to a native speaker of some language you’ve never heard before. Most people can’t even figure out where words start and stop, it’s all gibberish (someday let’s talk about glossolalia).

So language — any language — is just as symbolic a representation of reality as mathematics is. The symbols may be different (“±” rather than “plus or minus”) and that’s just a matter of translation.

So it occurred to me long, long ago that language — any language — could be symbolically represented by mathematical forms (and this gets into our first patent, a fun read in itself). All you needed was to know what mutually understandable information language — any language — was communicating.

Read that last sentence as “Determine the variables involved” and the mathematical forms pretty much reveal themselves to you.

I’ve done lots of simplifying on the following graphs. Our current Modality Engineering system does calculations in a 92 dimension Hilbert space and collapses that space as necessary. These statements could be collapsed to about a 30 dimension Hilbert space. Colors, directions, placement, angle, shape, …, everything has meaning in the following charts.

You are somebody that transmits confidence. And I have to admit that sometimes I'm concerned when talking with you as I know you'll know more facts than I will and it won't matter what we're talking about.

That first statement above, “You are somebody that transmits confidence. And I have to admit that sometimes I’m concerned when I want to convince you of something as I know you’ll know more facts than I will and it won’t matter what we’re talking about.” becomes an equation that generates this graph.

You have a knack for bringing out people's lack of self-confidence. People don't like to be corrected, even in private (I've never seen you publicly correct anybody) and when you correct someone (even in private) it's like being hit by a firehose. 'How does he know all this stuff?' People have faith in your being correct and in their being wrong.

The second statement, “You have a knack for bringing out people’s lack of self-confidence. People don’t like to be corrected, even in private (I’ve never seen you publicly correct anybody) and when you correct someone (even in private) it’s like being hit by a firehose. “How does he know all this stuff?” People have faith in your being correct and in their being wrong.” produces this graph.

Your commitment to the truth outweighs your commitment to people's feelings -- particularly when dealing with idiots or ?ssholes -- but it's a close race.

Statement three, “Your commitment to the truth outweighs your commitment to people’s feelings — particularly when dealing with idiots or ?ssholes — but it’s a close race.” generates this graph. It might look like statement 1’s graph and it’s not. But let’s finish with the basic charts first.

Everybody I know thinks you're a great guy. Everybody likes you. But everybody's intimidated by you, too, because they go around saying they're experts in something, you say you're not an expert in anything and you always know more about their field than they do so they end up feeling like imposters.

And finally, statement four, “Everybody I know thinks you’re a great guy. Everybody likes you. But everybody’s intimidated by you, too, because they go around saying they’re experts in something, you say you’re not an expert in anything and you always know more about their field than they do so they end up feeling like imposters.” looks like this (and it’s not the same as that generated by statement 2).


Ah, look. The Barre Town Commons. Beautiful, aren’t they?


And here’s the ice cream shop I told you about.


May I take your order?

I told you we'd stop for ice cream along the way


Everybody gets an ice cream when we go for our country rides. Aren’t you glad you decided to come along?

Okay, now everybody back in the car.

These four charts are mathematical representations of people’s experiences. In this case, of me. Let’s start putting these people in the same room. Let’s match people to the statements. Statement 1 is made by “A”, statement 2 by “B”, statement 3 by “C” and statement 4 by “D”.

A, C and I get together

What happens when A, C and I get together? A and C have similar and not identical “concepts” of me (this is demonstrated by colors, distance and relative positions from the axes, planar presentations, …). But — and this is the important But — if you were to change the scale of that image those two representations would intersect.

In other words, there would be certain topics, certain areas of discussion, certain activities that the three of us could participate in and have a great time.

But only certain topics, discussions or activities. And this is for A, C and me. Not A and me, not C and me, not A and C. It’s only for A, C and me. Vary from those certain topics, etc., and things get uncomfortable. The further things stray the more uncomfortable.

Let me give you a real life example; you get together with a friend and a friend of that friend (anybody picking up the social implications of this?). You’re all talking and chatting, maybe playing pool or darts, maybe in a theater line or at a game. Everything’s going great. Then your friend or your friend’s friend references something that was just between them and they laugh but you can’t because you’re not in on the joke.

They either have to bring you in on the joke to reestablish the social connectivity or they can continue down their road and you’ll feel more and more astray. Needing to stay “on topic” so that the social connectivity remains in tact is a real life example of the meaning of the A and C charts intersecting.

B, D and I get together

What happens when B, D and I get together? The first thing to notice is that B and D are projections from an origin point (not necessarily the same origin, though). This means both of their concepts of me recognize that there are things I won’t do, literally a point I won’t cross. It’s not so much a question of limits and boundaries as it is “I couldn’t imagine Joseph doing something like that”. A and C could easily imagine me doing anything on their planes of my existence but I couldn’t do anything off those planes of existence. B and D can imagine me doing lots more things than A and C can but B and D “know” there’s a point I won’t cross.

The next thing to notice is that B and D by themselves have similar colors in their charts of me (the dark areas at the top of both charts). This means they both and independently of each other have similar concepts of me. The fact that their charts merge and blend, some colors extending their range, some colors merging, is an indication that there’s a great deal we could discuss in common, do in common, that there’s not much the three of us could do or say that would cause any one of us to feel uncomfortable.

An example of this would be you, a friend and that friend’s friend getting together and truly hitting it off. Private or in-jokes don’t matter because there’s enough shared concepts that everybody laughs and laughs harder when details are explained. The comfort level always remains high.

A, B, C, D and I get together

What happens when the five of us go out for drinks or some such? Well, we shouldn’t. It’s as simple as that. A, B, D and I or B, C, D and I, yes, but the five of us — A, B, C , D and I — together?

Don’t even attempt it. Oh, we’ll be civil with each other, of course, but sooner rather than later A or C would find a reason to leave. Sadly, once A or C left the four remaining would suddenly get together fine. A or C, whoever was left, would put the blame on whomever left (non-consciously, of course), thinking the reason things picked up after the other left was because they were a poop of some kind.

The only other possibility is that A and C will focus on each other, B and D will focus on each other and I will have to “move” conversationally between them as both groups will want to pull me (again non-consciously) more in their direction.

Hmm…reminds me of my wedding…

Why is this so? Because the representations of A, B, D and I or B, C, D and I intersect but A, B, C, D and I don’t. The intersection of A, C and I isn’t even on the same scale as B, D and I, therefore such an encounter (in this limited scenario) would be doomed to fail.

What am I suppose to do with this, Joseph?

Let’s say we’re not a bunch of people going out for drinks or dinner or to the theater or even negotiating a contract (although using these principles for that makes things very fast and simple). Let’s say you’re putting up some marketing material (webpage, tv spot, YouTube video, social campaign, radio spot, print, …) and you know lots about your target audience…

…except How they’ll respond to it.

Oh, you have an idea, an opinion maybe, and as John Erskine said, “Opinion is that exercise of the human will that allows us to make a decision without information.”

But now you do know. With as much precision as you care to have (mathematically the 2nd and higher order elements can be forced to 0). All you need is some material from that target audience. Letters to the editor if you’re going for a newspaper ad on or offline, blog entries (comments and posts) they’ve written if you’re going purely online, snippets from a podcast or two if you’re going for radio time or snatches from a call-in show. Same for video and tv. You get the idea.

Want to sell a car to A and C? You now can know the single selling point — the intersection of the two plains — that will motivate them both. Or to B and D? You now know the constellation of factors — the merged colors in their projections — that will sway them.

Or to sell bleach. Or tv sets. Or cell phone plans. Or jams and jellies. Or new products they’ve never seen before.

So now you can know with mathematical precision and certainty how well your marketing efforts will be received (something NextStage calls Acceptance (and please, folks. Let’s not bastardize this concept as was done with engagement, okay?)), how long it will be remembered (branding), how often it will be talked about and passed on (something NextStage calls Viral Capacity), … and probably more importantly what, if anything, needs to be changed, by how much, in which direction, …

I was sent something today about a company measuring “intent“. I read through the material and couldn’t stop laughing my head off (hence my emPHAsis way above).

Hey, I make up words all the time. But at least when I make up a word, everybody knows I’m making it up. And I never take an existing word with an existing and reasonable definition and twist it to my own purposes.

For the record, I first wrote about NextStage’s Intender Status metric, a measure of when and how a visitor to a website would act upon the information presented, in an iMedia column, Usability Studies 101: The First Sale (is the Next Page) back in May ’05. We noted that research on this metric was nearing completion in July ’07 and demonstrated an application of the metric in Priming the Conversion Pump with Color in Aug ’07.

So, please, let’s leave Acceptance alone. NextStage’s Acceptance metric determines two things; 1) the mental attitude that something is believable and should be accepted as true and 2) that something is capable of being acceptable and accepted. The former is when we’re monitoring visitors to a site, the latter is one of the things many of NextStage’s tools do. Same piece of Modality Engineering, just depends what it’s looking at when it does it.

Of course, some time soon there will be a company saying they also measure “Acceptance” but what they mean by it is that the left flythrough of the subcutaneous click rate indicates hairs were accumulating on the last visit divided by 2.

Ice cream is so much better. Don’t you think?

Susan wants to sponsor a contest for the best redefinition of existing terms. As an example, redefine clickthrough to be something totally different yet completely Acceptable. Post your definitions as comments here.

(I told you she was the wicked one)

(but there’s homemade pizza and good, Canadian ale for the winner)


Probably the truly best part about all of this is that you never have to study these things, understand Modality Engineering beyond nodding appreciatively when I use the term, or even look at the types of charts I included above.

I mean, you know Susan‘s going to protect you from all that, right?

So why go through all this?

Much like Morgan Freeman’s Lucius Fox said to Christian Bale’s Bruce Wayne in Batman Begins when Wayne was recovering from the effects of the hallucinogen weaponized in aerosol form thanks to Fox’s antidote, “I just wanted you to know how hard it was.”


Posted in , , , , , , , , , , , , , , , , , ,