Design Content Research Insights

Groups Only HAVE Access TO Half of the RELEVANT Knowledge

Half of the Collective Intelligence in which the best possible goals and actions lie is never discussed.

Apparent goodness.

Groups earnestly assemble rational and compelling goals and actions from whatever material they are given.

The aesthetic appeal does not mean the design is sufficiently fit for purpose, feasible, or endorsed.

Research Insight Examples

These statistics may create unease and dissonance. But where feelings and inferences can be misleading, data explains reality.

This what is occurring, though most of it won't be experienced.
33–63%Never reaches the room

A group holds an average of 134 opinions about the decision in front of it, with 95% of groups falling between 110 and 190. Conventional group design practices surface 50 to 90. Between a third and two thirds of what the group thinks about its topic goes unsaid. Some because they don't feel safe to share it, some because the process doesn't ask for it.

Measured as: opinions surfaced in conventional dialogue, against the fully elicited set (mean 134; two standard deviations span 110–190). The fifteen measures below are what that shortfall looks like from five different directions.

Select any measure to read it. Use the arrow keys to move between them.
OPTIMAL25%50%75%100% lost
The Raw Materials
Opinions surfaced33–63%
Opinion accuracy41%
Opinions backed by data74–94%
The Dialogues
Intent read correctly95%
Reasoning made visibleNearly all
Cross-party visibility5–7 hidden
The Fit
System coverage20–50%
Outcomes endorsed5 of 13
Outcomes for others5 to 1
The Feasibility
Cost of Inaction6 of 30
Barriers mitigated5–8 of 35
Downsides foreseen0–2 of 13
The Alignment
Opinions agreed on76%
Agreement by categoryOutcomesSide-effects82 → 52
Leader's field of view1–3 of 13
Optimal — nothing lostShare of the attainable optimum lostRange across studiesSpread across categories
The Raw Materials · measure 1 of 3
33–63%never reaches the room

A third to two thirds of the group's material never reaches the room

With anonymity and the right prompts, groups express an average of 134 opinions — two standard deviations spanning 110 to 190, so 95% of groups sit in that band. Conventional dialogue surfaces 50 to 90 of them.

Measured as — opinions surfaced in conventional dialogue ÷ the fully elicited set (mean 134)

The Raw Materials · measure 2 of 3
41%of expressed opinions are wrong

Rational inferences, presented as knowledge

Four out of ten opinions key stakeholders express prove inaccurate when anonymously peer-reviewed. Not lies — reasonable inferences the group has no mechanism to test. They enter the dialogue as knowledge.

Measured as — expressed opinions surviving anonymous peer review ÷ all expressed opinions

The Raw Materials · measure 3 of 3
74–94%of what could be evidenced isn't

Four of the five opinion categories arrive with no evidence at all

Facts and figures on current state, trends and change drivers play a small role in how groups design goals and actions. Of the five categories of opinion, only assumptions have data collected for them at the outset — outcomes, means, barriers and unintended negative consequences are raised without evidence. Validation may be sought later for a barrier, but not at the beginning. Across an average 134 opinions, 5–25% carry validating data, against 90–95% that could.

Measured as — opinions with validating data ÷ total opinions used in goal and action design (avg 134), against a 90–95% attainable optimum

The Dialogues · measure 1 of 3
95%of negative judgments misread intent

Groups diagnose motive when the problem is information

When one party judges another's views or actions to be wrong, the judgment rests 95% of the time on a negative reading of that person's intent — that they are trying to achieve something they shouldn't. The research data indicates genuinely conflicting motivations occur less than 5% of the time.

Measured as — negative cross-judgments attributable to actual motive conflict ÷ all negative cross-judgments

The Dialogues · measure 2 of 3
Nearly allconclusions offered without their basis

Three reasons people disagree, and the largest one stays hidden

The largest by far is Different Data: one party holds facts and information the other does not, so the two draw different conclusions from what looks like the same question. Solving it has been problematic because few people will explain the reasoning behind their conclusions, in case that reasoning is found to be wrong. So the data stays private, and the disagreement is read as a difference of judgment, or of motive.

Measured as — conclusions offered with their supporting reasoning ÷ all conclusions offered

The Dialogues · measure 3 of 3
5–7 hiddenconstraints the other side never hears

Each party holds blockers the other does not know exist

In optimising customer/supplier relationships, BPO contracts, alliances, partnerships and joint ventures, the parties usually learn there are 5 to 7 assumptions, goals or constraints one party believes is holding the relationship back that the other party was entirely unaware of.

Measured as — cross-party constraints known to both sides before disclosure ÷ constraints held (5–7)

The Fit · measure 1 of 3
20–50%of the system goes undefined

The group describes only part of the system it intends to change

A group's goals and actions will change the current system of the topic at hand. Asked to define that system — the precondition for action completeness — groups identify 50% to 80% of it, against a desired standard of 100%. The remainder is changed without being described.

Measured as — portions of the target system the group identifies ÷ the full system, against a 100% standard

The Fit · measure 2 of 3
5 of 13outcomes endorsed. 8 disputed.

The target narrows to what nobody objects to

Groups average 13 outcomes that would indicate the plan had succeeded. They begin like-minded on 5, and rarely learn about the other 8 at all — so the target quietly settles at the lowest common denominator. This counts outcomes reaching full endorsement, which is a different measure from the depth of agreement shown under The Feasibility: groups agree strongly about outcomes as a category while endorsing few of them outright.

Measured as — outcomes reaching full endorsement ÷ total outcomes expressed (avg 13)

The Fit · measure 3 of 3
5 to 1outcomes favour the group

Groups name what they gain fluently, and what others gain barely at all

A group's proposed outcomes favour itself over its customers, citizens and member stakeholders by roughly five to one — WIIFM rather than WIIFT, what's in it for them. Measured against an expectation that a topic's target outcomes divide evenly between internal and external value, the external share ranges from none at all to half.

Measured as — external-party outcome suggestions ÷ total outcome suggestions, against an expected 50/50 split

The Feasibility · measure 1 of 3
6 of 30cost-of-inaction assumptions agreed

The business case rests on a baseline four fifths of the group does not share

The Cost of Inaction is the do-nothing option — the baseline any design is judged against when deciding whether to implement it, and the denominator of the business case. It is rarely zero. Groups express around 30 assumptions bearing on it, and reach like-mindedness on 6. The return on investment is then calculated against a starting point most of the group does not agree with.

Measured as — cost-of-inaction assumptions reaching full like-mindedness (6) ÷ cost-of-inaction assumptions expressed (30)

Where the theme lands

The group expresses 49 constraints and proposes 42 actions. It can describe why the plan will fail, and who it will harm, in more detail than it can describe the plan itself.

The Feasibility · measure 2 of 3
5–8 of 35barriers carry a mitigation

35 barriers exist. A Risks section names 5 to 8.

Asked directly, groups express 35 reasons why their goals and plans are likely to fail. A conventional Risks section names 5 to 8 of them, each with a mitigating solution. The other 27 are not rejected, and not judged acceptable — they are never raised, so nothing is decided about them at all.

Measured as — barriers surfaced and mitigated in the plan (5–8) ÷ barriers the group holds (35)

Where the theme lands

The group expresses 49 constraints and proposes 42 actions. It can describe why the plan will fail, and who it will harm, in more detail than it can describe the plan itself.

The Feasibility · measure 3 of 3
0–2 of 13downsides advocates can see

The people advocating the plan are least able to see who it harms

Among the reasons a group believes a plan will fail sits a distinct subset: damage the plan may do to others — unintended consequences, collateral harm, blowback. Groups express 13 of these on average. The people advocating the topic foresee none to two. The subset is small in volume next to other concerns, and it generates far greater resistance.

Measured as — downside concerns acknowledged by topic advocates (0–2) ÷ downside concerns the group expresses (13)

Where the theme lands

The group expresses 49 constraints and proposes 42 actions. It can describe why the plan will fail, and who it will harm, in more detail than it can describe the plan itself.

The Alignment · measure 1 of 3
76%of the field is contested

Three quarters of the opinion field is contested before anyone speaks

When groups anonymously rate their own opinions without attribution, they are like-minded on 24% and non-like-minded on 76% — walking into the first meeting.

Measured as — opinions reaching full like-mindedness ÷ all opinions rated

The Alignment · measure 2 of 3
82 → 52outcomes to side-effects

Agreement falls as attention moves from what they want to what could go wrong

Like-mindedness is not evenly spread. It is highest on the outcomes a group wants, and falls steadily through means and assumptions to barriers and unintended negative consequences — the two categories that determine whether the plan survives contact with reality.

Degree of like-mindedness, by opinion category
Outcomes82
Means76
Assumptions72
Barriers57
Unintended negative consequences52

Measured as — mean degree of like-mindedness per opinion category, scored out of 100

The Alignment · measure 3 of 3
1–3 of 13outcomes the leader can name

The person who convened the group sees the least of it

Leaders usually express 1 to 3 goals for a successful outcome. Asked privately, their groups express 13. The leader is not withholding the rest — they have not heard them, and the process gives them no way to.

Measured as — outcomes a leader expresses (1–3) ÷ outcomes the group expresses (13)

Restricted use.

Groups can only work with what they are given.
FAQ
Frequently asked questions
Why does SchellingPoint turn group decision-design activities, including conversations, into data?
Why does traditional group decision making (GDM) practice create false confidence?