While it’s clear that AI has the potential to transform market research, the real challenge for many researchers lies in integrating Artificial Intelligence seamlessly into their daily workflows.
Through our recent webinar in partnership with Greenbook, we explored our new AI Themes and AI Questions features that can help you simplify the analysis of large qualitative data sets, quickly identify insights and transform the way you engage with your data.
The Q&A that follows recaps many of the great questions that we didn’t have the opportunity to address live. To learn more about our current features and vision for Recollective AI click here.
If you missed the webinar live, you can watch the recording now!
How well can this handle group discussions, and the interactions among participants - e.g. who are agreeing with each other, disagreeing, elaborating on what someone else said, etc.
Recollective’s response:
When provided with the appropriate context, AI can effectively analyze group discussions to discern patterns of agreement, disagreement, and elaboration among participants. Recollective can and will implement several preprocessing steps to ensure the AI receives the necessary context, and this is an area that will continue to evolve to enhance the quality of the output. While AI models may still face challenges in fully grasping the subtleties of conversational cues, they are progressively improving in handling such complexities.
How does AI in this context handle potential coding challenges with sarcasm, irony, and cultural nuances when it comes to sentiment analysis and emotion?
Recollective’s response:
AI models, despite being trained on diverse datasets, can sometimes struggle with the subtleties of human language, making perfect interpretation of sarcasm, irony, and cultural nuances challenging. AI models generally perform better with languages that dominate their training set, such as English, Spanish, and French, though performance can vary across different models and as models improve. By offering verbatims—actual examples of how language was used in practice—researchers can validate and refine AI’s interpretations. This process helps identify and correct any misunderstanding, enhancing the accuracy of sentiment analysis and emotion detection.
How does the Recollective platform account for AI hallucinations? Outside of someone manually checking all of the applied filters to open ends, how would we know the AI applied it correctly?
Recollective’s response:
The Recollective platform manages AI hallucinations by directing the AI to generate answers strictly from the study data relevant to the asked question. Additionally, Recollective employs a process that extracts supporting verbatims, enabling the AI to double-check its work and providing researchers with direct quotes to corroborate the answers given. Furthermore, guidelines are implemented to ensure that responses are only provided when there is sufficient relevant information available from the study, enhancing the reliability and accuracy of the AI’s outputs.
Is this a GPT-based model?
Recollective’s response:
Recollective AI is agnostic regarding the AI models it employs. The platform continuously seeks the optimal balance of speed, performance, and cost relative to the specific AI function required. Currently, most of the AI technology used by Recollective is drawn from pre-trained models provided by OpenAI, which are securely accessed via Microsoft Azure across multiple geographic regions. This approach allows for flexibility in adapting to various AI advancements and needs.
Is the AI function available for all languages? Or only English?
Recollective’s response:
Recollective AI functions in all 27 languages that it supports, utilizing AI models that are trained in more than 100 languages. Performance tends to be better for languages that were more commonly included in their original training, such as English, Spanish, and French. Proficiency in various languages varies across different AI models, and it is continuously improving across the board. This capability allows Recollective AI to cater to a diverse global user base effectively.
Do you have a doc that summarizes security questions (ex.: where are the AI servers, etc.)
Recollective’s response:
Recollective provides comprehensive information about security concerns on our FAQ page, accessible online at Recollective AI page: www.recollective.com/recollective-ai. Additionally, for those seeking detailed insights into privacy and security measures, a Privacy Impact Assessment for Recollective AI is available upon request.
Can the AI analyze Recollective live qual video footage (from online depths / FG’s) into Themes & Categories too?
Recollective’s response:
The initial beta of Recollective’s AI features is currently focused on analyzing asynchronous responses, with plans to eventually include synchronous activities and discussions. This expansion means that AI Questions and AI Themes will eventually encompass all data collected in a Recollective study, including Live Group Chats and Live Video Interviews (IDIs). This broadening of capabilities will enable the AI to analyze live qualitative video footage from online depth interviews and focus groups into themes and categories as well.
Can this AI model consider segmentation in its summary and high-level insights output?
Recollective’s response:
Yes, researchers can incorporate segmentation in their summary and high-level insights output. Both AI Questions and AI Themes features support the use of segment filtering, allowing researchers to select a specific segment prior to asking questions. Researchers can then change the segment and re-ask the same question to compare responses across different segments. All questions asked are stored in a question history, facilitating easy comparison of responses from different segments. While current capabilities require the use of the provided segment filter rather than merely mentioning a segment in the question, future enhancements may introduce additional capabilities to simplify such comparisons even further.
Does this model allow you to edit or add to the themes, theme categories, etc. that are created by the AI?
Recollective’s response:
The current beta version of the AI Themes feature does not support the editing or merging of themes and categories created by the AI but it is under consideration for inclusion in a future update. This would provide researchers with more flexibility to refine and customize the AI-generated themes according to their specific needs and insights.
Do the Study objectives need to be spelled out before the research, or can they be added later? We’re 5/6 complete with our current project and didn’t populate the objectives.
Recollective’s response:
Study Objectives ideally should be established before the data collection begins, as this can significantly enhance the process of AI theme detection, which occurs concurrently with data reception. However, for AI Questions, the tool utilizes the current Study Objectives each time it answers a question, meaning it can adapt to any updates or changes made to those objectives even partway through a project. Therefore, while it’s optimal to set the objectives early, you can still add or adjust them as your study progresses, especially for functionalities that are responsive to real-time changes.
Do you have any tips on how to write a good Study Objective for the Recollective platform specifically?
Recollective’s response:
For crafting an effective Study Objective on the Recollective platform, it’s important to frame a clear and concise statement that distinctly outlines the goals or aims of your research. Here are some tips to ensure your Study Objective is well-crafted:
- Be Precise: Define what you hope to discover or learn through the study. This could be understanding consumer behavior, evaluating product reception, or any other specific goal.
- Include Context: Provide additional details about the participants, such as demographic information or specific attributes relevant to the study. Outline key research questions and relate them to the overall business objectives to ensure alignment with broader company goals.
- Guide the AI Analyst: Consider what information would be most beneficial for new research analysts reviewing the data. The objective should serve as a guiding light, offering insight into what is most important about the study.
Remember, the clarity of your Study Objective directly impacts the effectiveness of the data analysis process on the Recollective platform. It sets the stage for targeted insights and more strategic decision-making based on the research findings.
When our study has more than two languages, are verbatims translated into English or is the original language retained?
Recollective’s response:
When using the Recollective platform, the verbatims from AI Questions and AI Themes are pulled directly from the source responses and will be displayed in the original language they were provided in, whether it be French, English, or any other language. This ensures that you can see the exact words used by participants. However, the generated themes and answers from these verbatims will be presented in a single language, which is typically the primary language set for the study on the Recollective platform.
Can I ask to the AI about frequencies (how many people mentioned X Brand?)
Recollective’s response:
You can inquire about the frequency of mentions for a specific brand or topic using the Recollective platform, but the approach depends on the feature used. The Recollective AI Themes can provide the frequency of themes, categories, and emotions, which includes counting how often a specific brand is mentioned within those contexts. However, the AI Questions feature is less suited for direct quantitative queries like counting mentions, as it is primarily designed to understand the meaning behind a question to locate and summarize relevant responses. It’s important to note that these AI capabilities are complemented by Recollective’s existing analytical tools that support quantitative analysis. For instance, the Word Cloud feature can be particularly useful as it can be filtered to a specific task or segment and will count the number of instances a specific word, such as a brand name, is mentioned. This variety of analysis tools allows for both qualitative and quantitative insights based on the data collected in the study.
Can the AI analyze insights gathered from across multiple participants, taking part in standalone depth interviews?
Recollective’s response:
Initially, the beta version of these AI features is focused on asynchronous responses but there are plans to expand them to include synchronous activities and discussions. Our goal is to cover all data collected in a Recollective study, including data from Live Group Chats and Live Video Interviews (IDIs). Once in place, customers will have the ability to focus their analysis on a single Live Video Interview activity that contains multiple completed interviews.
What are the known current weaknesses in the research AI that we should be aware of?
Recollective’s response:
Recollective AI has some specific limitations that researchers should be aware of to effectively harness its capabilities:
- Quantitative Analysis Limitation: The AI Questions feature is primarily designed for qualitative analysis. It aims to understand the meaning behind a question to locate relevant responses and summarize this information. Therefore, it is not well-suited for answering quantitative questions directly, such as calculating percentages or frequencies.
- Comparison Challenges: For achieving direct comparisons between different data points or responses, it is more effective to ask separate questions and then compare the answers. We may improve this in time based on feedback.
- Question Framing: To maximize the effectiveness of the AI Questions, it is crucial to ask questions that are:
- Specific: Narrow down the question to target specific information (e.g., “What did participants like about [specific product]?” instead of a broad “What did participants like?”).
- Qualitative: Focus on qualitative aspects (e.g., “How do people feel about [specific topic]?” rather than “What percentage of people feel [a certain way]?”).
- Focus: Ask one focused question at a time to avoid confusion and increase the clarity of the response (e.g., “What are the benefits of [specific product]?” rather than asking for both pros and cons simultaneously).
- Segmentation and Filtering: Although the AI cannot process mentions of a person or segment by name within the question for filtering results, the platform does provide options within the interface to select specific individuals or segments. This allows researchers to focus the AI’s analysis on the selected group or individual, thereby tailoring the output to specific research needs.
Understanding these weaknesses allows researchers to better structure their inquiries and utilize additional platform tools to compensate, ensuring richer and more accurate insights from their studies.
Can we ask it to focus on certain tasks instead of the whole study? There is often a gradation of questions (spontaneous answers at the beginning and more focused questions at the end)
Recollective’s response:
Yes, both the AI Questions and AI Themes features on the platform allow researchers to concentrate their analysis on specific activities or tasks within a study. This functionality is particularly useful in studies where there is a gradation in the type of questions asked—from spontaneous responses at the beginning to more focused questions towards the end. By enabling researchers to select particular tasks or groups of tasks, the AI can provide more targeted insights relevant to specific aspects of the study. This feature helps in customizing the analysis to better meet the unique needs and objectives of each research project.
Can you describe the emotion analysis? Is it visual, audio or inflection-based?
Recollective’s response:
The emotion analysis on the Recollective platform is conducted solely through text or transcripts. This means that the platform detects emotions based on the written content of responses or transcribed material from audio and video sources. We have not yet focused on the analysis of emotions directly from audio or video elements, such as tone, inflection, or facial expressions
From a reliability and validity standpoint, what is the minimum number of interviews needed for the AI?
Recollective’s response:
From a reliability and validity standpoint, even a single interview can provide sufficient information for AI to generate insights, provided that the data is relevant and clearly articulated. The AI’s ability to validate its conclusions generally improves with additional data that supports or challenges the initial findings. Reliability refers to the consistency of the AI’s interpretations across similar datasets; therefore, a few interviews might not reliably represent broader trends or varied perspectives unless the context is highly specific and contained. Validity, which pertains to the accuracy and truthfulness of the AI’s analysis, will be limited when only a few interviews are available but its output will definitely be grounded in the interviews it has been provided.
Do you have sessions where we can learn more about the platform and how to use it at the highest level? Training, etc.
Recollective’s response:
Recollective provides custom training services for those seeking to enhance their proficiency with the platform. These sessions are tailored to help researchers understand and utilize the platform’s features at an advanced level. Additionally, Recollective offers extensive support resources, including a variety of how-to guides and helpdesk articles, which can be accessed at Recollective’s helpdesk (https://helpdesk.recollective.com/). These resources are designed to assist our customers in navigating the platform effectively and maximizing their use of its capabilities.
If you ask a question, but there are no mentions or not enough for a finding, will it tell you that?
Recollective’s response:
Recollective ensures that responses are provided only when there is sufficient relevant information available from the study. If a question is asked and there are no mentions or not enough data to support a finding, Recollective AI will notify the researcher of this shortfall. In such cases, researchers have the option to reframe their question or choose to relax any filters being applied, such as removing filters on activity, task, or segment. This functionality is particularly useful for quickly checking the presence or absence of specific elements like a brand name or topic. For example, if you ask, “Did anyone mention Brand A?” and Recollective indicates that no sources were found, you can be confident that “Brand A” was not mentioned in the responses, regardless of whether filters were applied or not.
.svg)
