Rich Insights

Rich Insights:
Brand Perception Inputs and Outputs

This post explains the types of activities and information that commercial team users are currently using (or training clients to use) to approximate answers for Brand Perception analyses.

Notes

  • In general, it would be helpful/beneficial for users (internal and external) to know what these tools are actually doing. For instance, the Company_high operator can be helpful for reducing result set to focus on a particular company, but it’s not clear exactly what is happening on the backend when you’re using it, which makes it hard to explain and use definitively (without trial and error).
  • 75% of the time this user is also looking at competitors in the same topic space they’re exploring for their own brand.
  • Adding a topic in addition to the name of the brand (e.g. “Apple” + “technology” vs. just “Apple") would likely yield better analysis results, but many of the Rosa users who are doing this by themselves are probably just using their brand name as the input.
  • Some of these users are also using Quid as their primary media monitoring tool.
 

 

Goals and Current Methods

Inputs (Search)

Find stories that are actually about my brand (narrowing, filtering out noise).

PROBLEMs:

Just because a brand is mentioned in an article, that doesn’t necessarily mean that the article is about that brand. In the case of Brand Perception, it’s important that the article is actually about the chosen company/brand.

CURRENT METHODS:

  1. (Search) Company operator
    1. Use this the most frequently because it reduces fewer articles than Company_high
    2. What is the difference (technically) between this and Company_high?
  2. (Search) Company_high operator
    1. Best for companies with high volume (Walmart, ATT, etc.) because it greatly reduces number of results, so might not be applicable for companies with fewer mentions
    2. What does this actually do? Does it essentially say “only include if Company is mentioned X number of times”?
  3. (Search) Title operator
    1. Used to specify that the name of the company occurs in the title of the articleBest for companies with high volume (Walmart, ATT, etc.) because it greatly reduces number of results, so might not be applicable for companies with fewer mentions
  4. (Search) Near operator / Abhishek’s “Proximity Operator Builder” Excel tool
    1. This is used for noise removal, but it’s more broad than using Company operator.
    2. Used primarily when doing a trend analysis. When “millennials” and “laundry” are within X# of words of each other, this can help you tell the article is more about this. 
    3. It’s used mostly when you’re looking at 2 things together: topic + topic, company + topic (e.g. “Apple" + “technology”)

 

Eliminating certain types of articles (narrowing, filtering out noise).

CURRENT METHOD:

  • (Search) use specific string of terms in their query:  AND NOT (stocks OR shares OR earning* OR “Market report” OR “Marketresearch” OR “research reports” OR “financial results” OR “analyst reports” OR “research and markets” OR “market forecast” OR “industry forecast”)
  • “NOT”-ing the Finance topic in the search topic aggregations might not make sense for financial institutions, like Citibank or Wells Fargo, etc.

1. Earnings Reports


CURRENT METHOD:

  • (Search) use specific string of terms in their query (??)

2. Press Releases


CURRENT METHOD:

  • (Search) use specific string of terms in their query (??)

3. Industry Projections

 

Outputs (Viz)

General Overview

CURRENT METHOD:

  • (Viz) Default network view
  • Rename
  • Identify most important/relevant clusters (How? What does this mean?)

Goal:

Get a general understanding of topics of conversation and identify the most important topics of conversation.

 

Find and Understand Key Entities

CURRENT METHOD:

  • (Viz) Search and tag companies
    • The bar chart is only primary mention and you lose a bunch of coverage because of this and incorrectly disambiguated entities

Goal:

Understand where key companies (e.g. my competitors) show up in the conversation.


CURRENT METHOD:

  • (Viz) Search and tag people
    • The bar chart is only primary mention and you lose a bunch of coverage because of this and incorrectly disambiguated entities

Goal:

Understand where key people (e.g. my company’s CEO) show up in the conversation.


CURRENT METHOD:

  • (Viz) Search and tag sources
    • The bar chart is only primary mention and you lose a bunch of coverage because of this and incorrectly disambiguated entities

Goal: 

Understand where key sources (e.g. sources or reporters I have a relationship with) show up in the conversation.


CURRENT METHOD:

  • (Viz) Look at Primary Mention bar chart to identify (then search and tag the important ones)

Goal: 

Identify unknown key companies (e.g. competitors).


CURRENT METHOD:

  • (Viz) Look at Primary Mention bar chart to identify (then search and tag the important ones)
    • Especially important for corporate comms group, they want to see where CEO is coming up

Goal: 

Identify unknown key people (e.g. my company’s CEO).

 

Understand Perception

CURRENT METHOD:

  • (Viz) Use network colored by sentiment to isolate negative articles. Tag the articles and put them in a bar chart to see which clusters they show up in.
  • (Viz) Put isolated negative articles in a scatterplot to understand if there’s a specific article or cluster that’s driving conversation
  • (Viz) Put isolated negative articles in a timeline to see the distribution and how far in the past/recent the article/cluster-in-question occurred

Goal:

Understand the most negative topics of conversation.


CURRENT METHOD:

  • (Viz) Use network colored by sentiment to isolate positive articles. Tag the articles and put them in a bar chart to see which clusters they show up in
  • (Viz) Put isolated positive articles in a scatterplot to understand if there’s a specific article or cluster that’s driving conversation
  • (Viz) Put isolated positive articles in a timeline to see the distribution and how far in the past/recent the article/cluster-in-question occurred

Goal: 

Understand the most positive topics of conversation.


CURRENT METHOD:

  • (Viz) Use a Social Sharing x Published Count scatterplot; pick out and investigate context surrounding clusters in the upper left
    • "For example, a class action lawsuit might be 4% of the convo, but it’s highly shared."

Goal: 

Understand what articles/topics are important to the general public, specifically content opportunities and potential risk areas.


  • Is this actually important to Brand Perception?

Goal: 

Understand what articles/topics are important to the media. (?)


CURRENT METHOD:

  • (Viz) Look at Primary Mention, Sources bar chart to identify (then search and tag the important ones)
    • [Rosas] will often have personal relationships with reporters and are interested in understanding who’s saying what.
  • (Viz) Use a network colored by cluster to look at the key themes surrounding a tagged group of “source” articles. Dig into articles to understand themes.
  • (Viz) Use a network colored by sentiment to look at whether the coverage surrounding a tagged group of “source” articles is positive, negative, or neutral
  • (Viz) Use a bar chart by cluster to understand volume of coverage broken down by topic for a tagged group of “source” articles
  • (Viz) Use a social sharing vs. published count scatterplot to understand which of the source’s articles are getting social media traction

Goal: 

Understand which sources are reporting on my brand most frequently and why.

 

Understand Specific Topic(s) or Interest Area(s)

CURRENT METHOD:

  • (Search) Adding key terms into the query?
  • (Viz) Search for a specific term or terms and tag it/them together as one group (e.g. “organic,” “organic food,” “organic foods,” “free-range,” etc. all searched and group together to approximate the concept of “organic foods”)
    • If they have specific messages they want to push, they want to see how often those terms are coming up. For example, whether, where, and how the concept of “organic foods” in appearing in conjunction with their brand.

Goal:

Understand whether, where, and how topics I care about are appearing in the larger conversation around my brand.

 

Shareable Quid: 4 Distinct Types of Sharing

Up until the point of this post, "Shareable Quid" has been used as a blanket term to refer to many different types of sharing of a Quid artifact that a user might want to employ—for example, sharing a link to a synthesized Quid deck on the web, embedding a link with an interactive Quid visualization into a news or blog site, etc. Based on what we already know about our current and target users' various business cases for leveraging Quid data and artifacts, we've identified 4 distinct types of sharing: Collaboration, Engagement, Reporting, and Publishing. It is important to sub-segment general "sharing" because each of the 4 types yields unique requirements, artifacts, and workflows based on distinct user needs.

All.png
 

 
COLLABORATE.png

Collaborate

Purpose

Active participation of colleagues on analysis

Time Period

Ongoing


Quid Artifact(s) 

Potential Features/Artifacts:

  • Share Projects feature
  • TBD

Current Features/Artifacts:

  • Share Projects 

User Roles

Receiver:

Contributes to the construction of outputs & insights to share with additional stakeholders

Creator:

Constructs the analysis to yield certain outputs & insights


Key Receivers (by Persona)

Stephanie

Colleagues

Rosa

Colleagues

Amber

Colleagues

Brian

Colleagues


Levels of Interactivity

Commenting:

  • Creator can comment
  • Receiver can comment

Degree of Interaction:

  • Interact in an exploratory manner

Editing:

  • Creator has full access to edit the analysis
  • Receiver has full access to edit the analysis
 

 

Engage

Purpose

Stakeholders interact with the pre-defined outputs and insights on an ongoing basis

Time Period

Ongoing


Quid Artifact(s) 

Potential Features/Artifacts:

  • Dashboard
  • TBD

Current Features/Artifacts:

  • None 


User Roles

Receiver:

Interacts with the data and draws her own conclusions from it

Creator:

Defines important outputs & insights (along with parameters) to be shared with receiver


Key Receivers (by Persona)

Stephanie

Self, Colleagues, Management, Clients

Rosa

Self, Colleagues, Lower level management

Amber

Self, Colleagues, Lower level management

Brian

Self, Colleagues, Management, Clients


Levels of Interactivity

Commenting:

  • Creator can comment
  • Receiver can comment

Degree of Interaction:

  • Interact in an exploratory manner

Editing:

  • Creator can edit
  • Receiver cannot edit 
 

 
REPORT.png

Report

Purpose

Creator shares synthesized & structured set of outputs & insights with the Receiver

Time Period

Discrete 


Quid Artifact(s) 

Potential Features/Artifacts:

  • Slideshow
  • Downloadable slide/doc
  • TBD

Current Features/Artifacts:

  • Storybuilder (limited)

User Roles

Receiver:

Consumes the creator's conclusions and makes business decisions based on it 

Creator: 

Synthesizes and creates the final output in a narrative format for a specific, known audience


Key Receivers (by Persona)

Stephanie

Self, Colleagues, Management, Clients

Rosa

Colleagues, Higher level management

Amber

Colleagues, Higher level management

 

Brian

Self, Colleagues, Management, Clients


Levels of Interactivity

Editing:

  • Creator can edit
  •  Receiver cannot edit

Commenting:

  • Creator can comment
  • Receiver can comment

Degree of Interaction:

  • Interact in a navigational manner
 

 
PUBLISH.png

Publish

Time Period

Discrete

 

Purpose

Creator shares synthesized & structured insights in a publication (news sites/blogs) or to promote thought leadership


Quid Artifact(s) 

Potential Features/Artifacts:

  • Embeddable link
  • TBD

Current Features/Artifacts:

  • None


User Roles

Receiver:

Passively consume data (with no outcomes)

 

Creator: 

Synthesizes and creates the final output in a narrative format to illustrate a point & catch the attention of a wide range of audience (content should be general and catchy)


Key Receivers (by Persona)

Stephanie

Clients (current and prospects), Industry-specific audience

Rosa

Not enough information

Amber

Not enough information

Brian

Clients (current and prospects), Industry-specific audience


Levels of Interactivity

Commenting:

  • Creator can comment
  • Receiver can comment

Degree of Interaction:

  • Interact in a navigational manner

Editing:

  • Creator can edit
  • Receiver cannot edit 
 

 

Quid User Personas

 

Understanding Search

“Understanding Search” is a series that I wrote while I worked at Quid. It explores Search as a technical discipline from a layperson’s perspective.

These posts are written for a non-engineering audience by a non-engineer author—with an incredible amount of research and support from engineers who specialize in search technologies, obviously. They explain aspects of how Search works—both in the context of Quid and as a standalone academic discipline—so the broadest-possible audience can understand the implications of future Search-related changes to our current product.

These posts are not written (primarily) for a highly technical audience. There are liberties taken, including certain omissions and oversimplification, for the sake of understanding by the intended audience. These liberties have been vetted thoroughly with search-focused engineers.

These posts also don’t pretend to cover even a fraction of the collective knowledge available about the academic discipline of Search. The topics covered have been explicitly chosen to illuminate the past, present, and future of Search at Quid; even in regards to Search at Quid, they are admittedly non-exhaustive.

 

Understanding the Data Acquisition Process

For Quid's News and Blogs Dataset


Overview

Before news articles can be searched and visualized in Quid, they need to be ingested and formatted to fit a certain structure that makes them usable across the Quid pipeline. This post explains the data acquisition process for Quid’s News and Blogs dataset, which includes the sourcing and structuring of news data for system-wide use.

Figure 1.0: Quid Data Acquisition Process Diagram (as of July 18, 2017)

 

Step-by-Step Process Walkthrough

  1. The acquisition process starts with the Fetcher, which fetches documents (a.k.a. “articles”) from the Moreover [1] API in the form of raw XML files.
     
  2. We create and store a backup of each of these raw XML files.
     
  3. The documents are put into a Kafka topic [2], which is basically a container that holds and enqueues the documents for processing. This first one is called “Kafka Fetched” because it holds all the documents after the fetching phase.
     
  4. The Parser consumes messages in order from the Kafka Fetched queue and handles parsing the raw XML.
     
  5. During the parsing, we take the unstructured XML file and pull out all the fields that Moreover provides—this includes the article title; article body text; and other metadata, like source, published date, and source URL. Then we reorganize that information to fit a specific, pre-defined structure (schema).
     
  6. The output of parsing is a structured (parsed) JSON file [3]. This structure is the same across all documents Quid uses, and these structured documents can be used across the Quid pipeline.
     
  7. The parsed JSONs are put into the Kafka Parsed topic, where they are enqueued for Annotations.
     
  8. During the Annotations phase, we enrich the document with additional fields, called annotations. The Annotator calls out to the annotations service with a piece of text and in return receives the extracted entities and keywords.
     
  9. The annotated JSON files are put into the Kafka Annotated topic and enqueued for further use. This is, in essence, the end of the data acquisition phase.
     
  10. This final structured, annotated JSON file is called the “News Article [4].” This structure is the same across all documents Quid uses, and these structured documents can be used across the Quid pipeline. Depending upon where the document is being used in the pipeline, it undergoes further transformations. (We’ll explore some of these transformations in subsequent posts.)
 

FOOTNOTES:


[1]  Moreover, part of LexisNexis, is the news aggregation service we use to source our news and blogs dataset. Fun fact: Quid gets ~500 documents from Moreover every 20 seconds.

[2]  You’ll see Kafka topics appear several more times throughout this diagram. For the sake of this explanation, you can think of a “topic” like a container. In each case, the Kafka container is named for the phase the documents it holds has just exited. It is helpful to have a separate Kafka container for each phase in the pipeline and name it accordingly because, if something goes wrong, it is easier for the engineer to identify exactly where the problem is occurring.

[3]  For the sake of this explanation, you can think of this parsed JSON file as a blank template with labeled input fields for specific information. Some of this information is filled in during the parsing phase (article title, body, metadata), and some will be filled in during subsequent phases (e.g. entities, keywords).

[4]  This object is technically called the “Searchable News Article,” but there is also another artifact that Search uses later on in the pipeline that is also called the “Searchable News Article” because it is actually what is being referenced when a user performs a search in Quid. Unfortunately, the artifact in the Acquisition phase couldn’t be renamed in the system, but for the sake of this explanation the Acquisition phase artifact is called the “News Article.” This is confusing but a technicality that could be important in case you hear reference of the “Searchable News Article.”

Developing User Models

What is a User Model?

An excerpt from the "User modeling" Wikipedia post:

"User modeling is the subdivision of human-computer interaction which describes the process of building up and modifying a conceptual understanding of the user. The main goal of user modeling is customization and adaptation of systems to the user's specific needs. The system needs to "say the 'right' thing at the 'right' time in the 'right' way". To do so it needs an internal representation of the user."

In order to better serve users with data that is relevant to their needs, we first need to collect information about and understand the users' needs. User models are a tool we can use to develop an understanding of each user by gathering information about her and using it to inform our system's functions.

User Classification

I used information from the "Who's TOHU?" Product Brown Bag presentation to create a first draft of user classifications, which we can use as a starting point for developing user models in the system. The classifications are a continual work in progress; please comment on them, particularly to point out gaps in coverage or to add examples of Quid users (present, prospective, or churned).

** and * = We have user interviews for this user from 2016 and 2015, respectively

Agency / Consulting

CPG / Retail

Government

Healthcare

Industry

Technology

Manual Entity Disambiguation

Overview

Entity disambiguation is the task of properly identifying and linking the identity of entities mentioned in text. In an ideal world, entities would be perfectly identified every time. However, until that day comes, there will be some circumstances in which entities are not combined according to a user’s preferences.

Currently there is no way for users to manually combine entities they have identified as erroneously un-combined by the system or delete (hide/filter out) “bad” entities. Additionally, Quid has no feedback mechanism that would allow us to learn from these combinations and deletions.

By now, this is a well-known issue for our users. Below I’ve further described the specific pain points related to this problem:

 

Pain Points

  1. Entities (people, companies, products, etc.) that are actually the same have not been properly combined, and there is no way for a user to manually repair these combinations.

    1. Examples:

      1. Companies: “Apple” vs. “Apple, Inc.”

      2. People: “Mehmet Oz” and “Dr. Mehmet Oz” and “Dr. Oz”

        1. **Titles (Jr., Sr., President, Count, Dr.) are a big problem, so this happens for people a lot

    2. Affects: primarily News and Blogs dataset

    3. Implications:

      1. “Top* companies” or “top* people” mentions are not necessarily correct if entities are improperly combined

        1. *We don’t currently have a “top” algorithm. “Top” is a label to show the top of the list of entities, so the interpretation as “top” is accurate 80% of the time; there are some case in which it’s wrong

      2. Users have to remake bar charts in Excel, using Quid data but manually combining necessary entity values

      3. Search results may be affected; if entities aren’t combined properly, a user may not get all the results for a given entity

      4. Users become frustrated with, mistrustful of data quality
         

  2. Some entities are just “bad.” Currently, the only way to remove things is to filter them out; there is no deleting.

    1. Examples:

      1. Quid did a competitive analysis of several airlines, and there were certain things they just didn’t care about, wanted to easily delete/hide/filter out

      2. “Reuters” is marked as an entity but it was actually the republisher of the article, not the topic

      3. A company is identified as a person (or vice versa)

      4. An acronym is mapped to the wrong entity (e.g. “IoT” as “Institute of Transportation” instead of “Internet of Things”

      5. A phrase is considered a unique entity (e.g. “Michael Douglas Barack Obama” from a list of people)

      6. The statistical method failed (a sentence like “In France, democracy was founded in 1789” could trick the algorithm into marking “democracy” as a company)

    2. Affects: overall data quality

    3. Implications:

      1. The article is the filtered unit, not the entity. Filtering out entities actually filters out the entire article the entity is associated with, so if other entities are also associated with that article, they are affected.

      2. Article may still contain relevant / accurate entities, but it just wasn’t extracted out as the “top” one.  
         

  3. There is no “learning” related to improperly combined/differentiated entities. If something was fixed once by a user, it won’t be fixed the next time; they would have to make the same combination (or separation) again. (These fixes could be “learned” on a user basis, an account/client basis, and/or a global basis)

    1. Affects: primarily News and Blogs dataset

    2. Implications: Users will become frustrated if they have to keep making the same corrections every time
       

  4. Crunchbase and CapIQ (our sources for the Companies dataset) may have overlapping companies that we missed when doing the initial entity combination.

    1. Affects: primary Companies dataset

    2. Implications: The number of improper company combinations may be even higher than it was, and there’s currently no way for a user to fix this
       

  5. In Opus, unless entities are phrased exactly the same, they won’t be combined.

    1. Affects: Opus users

    2. Implications: Users can’t manually correct these combinations

 

Open Questions and Considerations

Questions

  1. Should users be able to separate entities that they combine? How?

  2. How will users know which entities are the result of a manual combination? How will they know which entities were combined to achieve this ultimate entity?

  3. At what point in the Quid workflow should we allow / prompt users to start assessing and combining entities?

  4. At what level should users be able to combine entities?

    1. Individual chart

    2. Entire network

    3. Globally, always per user

    4. Globally, always per client

    5. Globally, always for all users

  5. When are users after a categorization change (e.g. subtypes like “tv personality” vs. “business person” vs. “politician” for Donald Trump) vs. a full merger?

  6. We currently only expose some entity “classifications” that Alchemy provides—People, Company, Institution, Location—but there are others available, which could be highly relevant depending on the use case—Field Terminology, Drugs, Health Conditions, Products. How can the classifications we show evolve to support various use cases?

  7. If a user were to delete a node, it would affect all the entities contained in that node (article). Since the node/article is currently the focal point of activity, if a user would filter or delete an entity*, they’d be filtering or deleting the whole node/article. Is there a way to “delete the entity” by deleting the record of mention or significance of the entity from the node/article rather than deleting the whole node/article itself so the other entities in that node/article remain unaffected?

    1. *A user may want to delete an entity if it is badly tagged or not relevant to the user’s analysis

Considerations

  • Alchemy (News/Blogs, Companies, Patents) and Basis (Opus) are services we use for entity extraction (and top keywords). They do the entity disambiguation (except in Opus; we don’t have disambiguation there yet)

  • Our sums (e.g. “Top Companies”) are based only on Primary Mentions, not on All Mentions. We can only access All Mentions for an entity by:

    • Filtering by the entity and “any mention”, BUT there’s no way to select this group

    • Finding the entity via search, BUT if entities aren’t de-duplicated, then it doesn’t show everything that’s applicable

  • For each article’s entities, we only show the top X number; there are no thresholds. So, even if entity 5 and entity 6 are scored the same, entity 6 is cut off if we only show the top 5.

  • We only have graphs (bar charts and timelines) that show “Top” entities mentioned, not “Any”/”All” mentions of an entity because there would be duplicate representation of the same article

    • One potential solution for this is to make the bars of an “Any”/”All” graph unclickable, so the user can see the number of mentions but not select an article/node in the bar

 

MVP Design Proposal

Current design specs can be found via this Invision link: https://quid.invisionapp.com/share/M8C61O0FH

 

Future Versions Design Proposal

 

LIXIL Card Sorting Exercise Analysis

Overview

This analysis is part of a larger user research card sorting exercise focused on understanding which research activities are important to users based on their role or job function AND how users logically group these research activities. We asked LIXIL to participate in this exercise; below is the feedback an assessment of their collective results, shown in aggregate and broken down my teams that participates (5 total, each comprised of members in the same job function).

Highlights

Which research activities were most important per group?

 

  • Understand consumer trends

  • Understand a company or market

  • Analyze and measure conversations around trends, products, events or issues

  • Inform and optimize brand message and delivery

  • Predict future consumer trends and needs

Headquarters


  • Understand a broad topic at a high level

  • Understand a specific topic at a high level

  • Understand an industry at a high level

  • Understand a company or market

  • Identify my competitors

  • Identify my competitors' current and future trends

  • Understand my competitors' business strategy, products, vendors, etc.

CIG


  • Understand consumer trends

  • Analyze the reputation of a person

  • Compare my products with my competitors' products

  • Identify my competitors' current and future trends

  • Understand my competitors' business strategy, products, vendors, etc.

  • Compare patents with my competitors

  • Understand intersection and overlaps of technology

  • Understand capabilities of a certain technology or material

Research and Development


  • Understand a specific topic at a high level

  • Understand consumer trends

  • Understand a company or market

  • Analyze the reputation of my competitors

  • Identify my competitors

  • Compare my products with my competitors' products

  • Understand my competitors' business strategy, products, vendors, etc.

  • Compare patents with my competitors

  • Identify technology gaps

  • Understand capabilities of a certain technology or material

  • Predict future consumer trends and needs

LHT


  • Understand a specific topic at a high level

  • Understand major events or activities

  • Understand a company or market

  • Identify my competitors' current and future trends

  • Understand my competitors' business strategy, products, vendors, etc.

  • Compare patents with my competitors

  • Understand capabilities of a certain technology or material

  • Predict future consumer trends and needs

  • Analyze product reviews

LWT

 

 

Which research activities were most important overall?

 

None

5/5 Groups (100%)


  • Understand a company or market

  • Understand my competitors' business strategy, products, vendors, etc.

4/5 Groups (80%)


  • Understand a specific topic at a high level

  • Understand consumer trends

  • Identify my competitors' current and future trends

  • Compare patents with my competitors

  • Understand capabilities of a certain technology or material

  • Predict future consumer trends and needs

3/5 Groups (60%)


  • Identify my competitors

  • Compare my products with my competitors' products

2/5 Groups (40%)


  • Understand a broad topic at a high level

  • Understand an industry at a high level

  • Understand major events or activities

  • Analyze and measure conversations around trends, products, events or issues

  • Inform and optimize brand message and delivery

  • Analyze the reputation of my competitors

  • Analyze the reputation of a person

  • Understand intersection and overlaps of technology

  • Identify technology gaps

  • Analyze product reviews

1/5 Groups (20%)


  • Understand media coverage of products

  • Understand global and local issues

  • Predict future trends

  • Assess product trends

  • Analyze the reputation of my company

  • Understand my competitors' media strategy

  • Map my competitors' acquisitions and investments

  • Analyze a survey

0/5 Groups (0%)

 

High Frequency Cards in Standardized Groups

I found some patterns in some of the group names and their contents, so I created a set of standardized groups to combine groupings that were similar across the LIXIL team results. (Not all the LIXIL-created groups were able to be combined together; there are some "outlier" groups, which means there were certain cards the team did classify in a consistent way from group to group)

  1. Competitive Analysis (8 unique cards, 85% agreement)

  2. Marketing (10 unique cards, 75% agreement)

  3. Analysis (7 unique cards, 71% agreement)

  4. Technology (6 unique cards, 61% agreement)

  5. Trends (8 unique cards, 58% agreement)

  6. General Understanding (13 unique card, 55% agreement)

  7. Survey (1 unique card, 100% agreement)

Standardized Groups

 

Most Frequently Occurring Cards in Each Group

This list shows groups of cards that were most frequently placed in each of the standardized categories, which shows high agreement between groups on classification of these particular cards. (Check out the Standardization Grid at the bottom of this section to look at the distribution of cards across the standardized categories.)

 

  • Identify my competitors' current and future trends

  • Understand my competitors' business strategy, products, vendors, etc.

  • Map my competitors' acquisitions and investments

  • Identify my competitors

  • Understand my competitors' media strategy

  • Analyze the reputation of my competitor(s)

  • Analyze the reputation of my competitor(s)

  • Compare patents with my competitors

Competitive Analysis


  • Understand consumer trends

  • Predict future consumer trends and needs

  • Analyze the reputation of my company

  • Analyze and measure conversations around trends, products, events or issues

  • Understand media coverage of products

Marketing


  • Analyze the reputation of a person

  • Analyze product reviews

  • Analyze a survey

Analysis


  • Identify technology gaps

  • Understand capabilities of a certain technology or material

  • Understand intersection and overlaps of technology

Technology


  • Predict future consumer trends and needs

  • Analyze product reviews

  • Predict future trends

  • Understand consumer trends

  • Assess product trends

Trends


  • Understand major events or activities

  • Understand a specific topic at a high level

  • Understand an industry at a high level

  • Understand a broad topic at a high level

  • Understand global and local issues

General Understanding


  • Analyze a survey

Survey

 

Standardization Grid

The standardization grid shows the distribution of cards across the standardized categories you have defined. Each table cell shows the number of times a card was sorted into the corresponding standardized category.

 

Standardization Grid.png

 

Groups of Cards that Appeared Together

Although some cards were hard to classify, there were certain cards that always or often appeared together. So, even if the teams had different opinions about which category a set of cards should be in, there were some sets of cards that were always found in the same category. This means that the participants felt like these individual research activities logically belonged together, regardless of their larger category or the other cards they were paired with. (Check out the Similarity Matrix at the bottom of this section to look at all the individual card parings).

Always (100%)

  • Understand a broad topic at a high level

  • Understand a specific topic at a high level

  • Understand an industry at a high level

Set 101


  • Compare my products with my competitors products

  • Compare patents with my competitors

Set 102


  • Identify my competitors

  • Identify my competitors' current and future trends

  • Understand my competitors' business strategy, products, vendors, etc.

  • Map my competitors' acquisitions and investments

Set 103

 

Often (80%)

  • (Set 101) +

  • Understand major events or activities

Set 201


  • (Set 103) +

  • Analyze the reputation of my competitor(s)

  • Understand my competitors' media strategy

Set 202

 

Similarity Matrix

The similarity matrix shows the percentage of participants (teams) who agree with each card pairing.

 

Similarity Matrix.png
 

Packaged View Plus: Presentation Analysis

In an effort to understand which visualizations and insights are most frequently included in client-facing work, I did an analysis of presentations created by the commercial team as part of a larger research initiative focused on re-shaping Quid for use by the TOHU. I examined a sample of 9 total presentations created by the commercial team for clients, taken from the repository of presentations called “Quid’s Quest for Quintessential Content." All 9 were classified as “News Narrative” because these are exclusively based on the news and blogs dataset, which is the dataset used by the most users and has the fewest data quality challenges. I wanted to make sure a mix of different industries were represented, so there are at least two each for Consumer Products (3), Tech (2), Travel (2), and Communications / PR (2). For the purpose of the analysis, I used the commercial team not as a proxy for all end users but as a proxy for an expert Quid analyst.

My goal was to see which combinations of configurations appeared most frequently at a high level, but using something like a coded spreadsheet was insufficient for these purposes because each visualization has so many configuration variables. Instead I used a data visualization system inspired by a project called "Dear Data" to code each slide in the presentations. Below you'll find an example, a full list of coded presentations and individual slides, and the coding key.

 

Example Coded Presentation

Internet of Things Narrative


This set of characters represents the slides in a News Narrative about the Internet of Things. Each of the shapes and it’s vertically stacked elements represent a single visualization. Each of these shape groups represents a single slide. So, as you can see, some slides contained multiple visualizations.

 

Coding Key

I ended up choosing this coding method because there were so many variables to encode that using a traditional method, like a spreadsheet or list, was more confusing than it was helpful. Using visual iconography allowed me to process and find patterns in robust amounts of information about multiple objects at the same time.

PP-Preso_06.jpg
 

Individual Presentations

Each of the following images represents a full presentation's worth of visualizations, shown in the order in which they appear in each presentation. Looking at a presentation's slides in order allowed me to understand the narrative arc of each and compare narrative arcs across presentations.

 

Aggregated Slides

Aggregated by Visualization Type


Aggregating the visualizations by type across all presentations allowed me to identify patterns in the most frequently used visualization types. (For views of the same type that appeared back-to-back within a presentation, I maintained their ordering in the aggregation and represented this relationship with a bracket. This occurred frequently enough to call it a pattern; this pattern helped me understand that often users were drawing a single point across multiple slides by changing a single configuration element from slide to slide.)

PP-Preso_09.jpg
PP-Preso_10.jpg

Aggregated set of visualizations

PP-Preso_08.jpg

List of most frequently used slides

 

Key Takeaways

  • Summary slides at the beginning of the presentation. 5 presentations began with a slide that contained a bulleted list of overall takeaways or recommendations from the analysis. They varied in name— “Executive Summary,” “Key Insights and Recommendations,” “Summary”—but served the same purpose.
     
  • Multiple Quid visualizations on the same slide. 6 presentations had 1+ slides with multiple (2-3) Quid charts on the same slide to show a comparison of some kind. Of 79 total slides coded, 17 slides (21.5%) fall into this category. All but one of these slides contained multiples of the same viz type: bar charts (6), networks (6), or timelines (4)
     
  • Custom visualizations. 3 presentations had 1+ slides with text or a visualization (Excel chart, flowchart made of boxes, etc.) based on Quid data, but not using a Quid data visualization. Of 79 total slides coded, 10 slides (12.7%) fall into this category.

Packaged Views Plus

A Frontiers presentation was given on this topic on January 24, 2017. The presentation used is available below and may provide some additional context on this project's research methodology. There's also another full blog post dedicated to the research for this initiative: "Packaged Views Plus: Presentation Analysis."

 

What are Packaged Views?

  • A View is a visualization plus a set of configurations that yield a particular type of insight(s) for a user
  • Packaged Views are a selection of views that Quid recommends to a user for consideration and analysis because they are likely to be relevant to the user's interest

The concept of Packaged Views exists in the current version of Quid. However, there are only 5-6 Packaged Views per data set; we don't promote them as a primary means of navigating the Visualization section; and, besides the Packaged View names and tooltips, they don't provide much context for why a given view could be valuable to a user. Our intent with the new iteration of Packaged Views is to allow users to find insights more quickly by:

  • Promoting Packaged Views as an approachable, primary means for understanding a network and deriving insights by increasing their prominence in the UI navigation
  • Providing a robust set of views that, in theory, would allow the user to find all the insights she needs to build a "story" or create a sharable artifact
  • Providing additional context for what insights can be derived from each view and why these insights may be relevant to a user's interests 

Types of Packaged Views

While doing our research and analysis, we determined that a single view does not always yield a single insight. Most views can be used to derive multiple insights, and, more interestingly, users often compare 2 or more views—either versions of the same view or different views all together—to find a single insight. Additionally, some views that are highly valuable and appear over and over again in final presentations are based on a single view but require a level of customization by the user (e.g. filtering, tagging). We categorized the different types of views as follows:

  • Basic
  • Interactive
  • Comparative

Basic

  • The user can derive valuable insight(s) from a single view with its default configurations
  • 1 View = 1 or more insights

Interactive

  • The user is required to manipulate some configurations of a view in order to derive the ultimate insight(s), OR
  • The user is expected to have "advanced" knowledge on how to analyze a given view to yield a particular type of insight(s)
  • 1 View + Configuration = 1 or more insights

Comparative

  • The user reaches insight by drawing a comparison between 2 or more views
  • The views being compared many require configuration (e.g. filtering a network map by individual companies and comparing the different outputs)
  • Often these views are pared with a Basic View to draw a particular type of insight
  • 2 or more Views (+ Configuration) = 1 or more insights

Potential Evolution of Packaged Views

Basic Packaged Views

For the MVP, we have compiled a list of Basic Packaged Views that are most frequently used (for the News and Blogs dataset). We will present this same set of Packaged Views (per dataset) for every query. As we gain the ability to collect more information about users' intents (i.e. preset questions / use cases), we will tailor the set of Packaged Views a user is served based on what is most relevant to a given use case.

Interactive and Comparative Views

Follow-up iterations of this feature could be to add guided experiences, like using a wizard, for drawing insights from Interactive and Comparative Views.

Process for Selecting Packaged Views

  • Started with the News and Blogs dataset since this is the most commonly used dataset and chose to focus on the most common "use cases" that are informed by News and Blogs: Narrative Mapping and Competitive Intelligence
  • Analyzed past presentations created by the commercial team to find the most frequently used views
  • Used information from a workshop with the commercial team (conducted by Yael and Ben) that identified the most valuable views and the insights each can yield
  • Created a comprehensive list of views and divided each into Basic, Interactive, and Comparative categories
  • Assessed Basic Views for those most frequently used and deemed most valuable (based on feedback from the commercial team)
 

Recommended Set of Packaged Views (MVP)

 

 

 

Other Views That Were Considered for Packaged Views

These views were also assessed and deemed valuable by members of the commercial team. Based on frequency of use and degree of value, we determined the following views will not be served to the user as Packaged Views—at least not in the MVP where every user is served the same set of packaged views per data set. (Users can still manually configure any of these views should they be deemed necessary)

Basic Views

 

Interactive Views

 

Comparative Views

 

 

List of Insights for Each Packaged View

(News and Blogs Dataset)

 

TOHU Vision Development

As a part of a vision development exercise, we have come up with a workflow tailored to suit the quintessential '1 hour user'. With an aim to reduce the time it takes to arrive at valuable insights to minimum, we imagined a journey that takes the user from pre defined use cases to sharable insights just in a couple of steps. Users have the option to delve deeper an curate their story, segments and search if they want. 

TOHU-1.jpg

 

Initial Workflow Concept

1. Preset Use Cases

 

2. Choose a Use Case (Ask My Question to Quid)

After selecting a use case, the user fills in the information specific to their query.

 

3. Arrive at Insights (Shareable Quid)