How We Benchmark Best Practices
Starfleet Research benchmarks best practices in technology-enabled business initiatives. We study how organizations combine technology, workflows, people and data, and compare relevant peers to identify practices associated with stronger performance. Our findings help organizations assess where they stand, decide what to improve, and measure whether the changes they make are delivering better results.
There is no single survey instrument or performance metric that applies to every technology market. Each study is designed around the initiative being examined, the practitioners best qualified to address it and the outcomes that matter in that operating environment.
How We Move From Evidence to Guidance
The exact research design varies by study, but the analytical progression is consistent: define the question, gather relevant market evidence, compare organizations in meaningful ways and translate the differences into practical benchmarks and decision guidance.
How to Put a Benchmark to Work
Starfleet Research provides peer benchmarks and evidence about practices associated with stronger results. Your team can use that evidence to choose what to test and then measure progress in its own operation.
- Set a baseline.Choose a KPI tied to the business goal. Record current results and how and when they were measured.
- Compare with relevant peers.Look for a meaningful gap. Check whether the peer group and the way it measures performance fit your situation.
- Choose a change to test.Prioritize a practice that addresses the gap. Assign an owner, set a target and agree on when to review results.
- Measure, adjust and repeat.Use the same KPI to assess the change. Decide whether to adopt it, adapt it or stop, then reassess your benchmark as conditions change.
From a research question to an operating decision
Define the outcome and compare practices. A study could compare average absolute percentage error in weekly sales forecasts over 12 weeks among restaurants of similar format and scale. It could then examine how often teams actually reviewed and revised forecasts in groups with lower and higher reported error.
Test and reassess. An operator could use the same error measure to establish its own baseline, pilot a weekly forecast review and measure the next 12 weeks. The team could consider holidays, promotions and other changes in trading conditions before deciding whether to keep or adjust the review schedule.
This example illustrates the method. It does not report study findings or assume that more frequent reviews improve accuracy.
How Studies Are Built
A credible benchmark begins with the right respondents, the right questions and a clear understanding of what is being measured. Research design is therefore tailored to the technology category, industry and business issue under examination.
Survey participants are selected for relevant experience with the subject being studied. Depending on the project, that can include technology buyers and users, owners and operators, functional leaders, IT and cybersecurity professionals, general managers, directors and other practitioners.
Qualification criteria are designed around the research question rather than around a generic panel definition.
Starfleet Research surveys commonly use structured questions to measure adoption, priorities, practices, challenges, capabilities and outcomes. Open-text questions may also be used where practitioner perspective can add useful context.
Question design aims to make comparisons possible while preserving the distinctions that matter within a specific market.
Results can be analyzed across relevant respondent and organization characteristics, including role, company size, number of locations, business format, geography, operating model and technology maturity.
Segmentation helps distinguish meaningful patterns from broad market averages that may conceal important differences.
Primary survey findings may be supplemented with secondary research, market information and subject-matter analysis to provide context around the technology, business environment and operating challenges being studied.
When external data is used, it should be distinguished from proprietary Starfleet Research survey findings.
See How the Research Starts
Every Starfleet Research study begins with a structured research instrument designed around the business questions being investigated and the practitioners best qualified to answer them. The questions below illustrate how a restaurant technology study translated operating challenges into measurable variables that could later be compared across respondent segments.
Effective research design moves beyond asking whether a technology is being used. It examines what organizations are trying to improve, how valuable specific capabilities may be, where current processes fall short and which outcomes matter most.
Those responses can then be analyzed against organizational and respondent characteristics to identify differences in priorities, maturity and operating practices.
Example of How a Research Sample Is Structured
In one targeted online survey, Starfleet Research collected responses from 373 qualified restaurant professionals to examine how operators across multiple formats are using technology to improve performance, efficiency and decision-making. The sample was intentionally diverse across geography, leadership role, restaurant type and company size.
Geographic Mix
- North America69%
- Europe15%
- Asia7%
- Other Regions
Latin America, Middle East, Australia/New Zealand9%
Leadership Role
- Owners, Founders, C-suite55%
- Operations, Technology & Finance Leadership
Including IT managers, operations leaders and general managers36% - Advisory or Technical Support9%
Restaurant Format
- Quick-service41%
- Fast-casual29%
- Casual dining16%
- Full-service8%
- Bars, cafés & specialty concepts6%
Together, the respondents provide a cross-format view of how restaurants are adapting their technology strategies to navigate rising costs, operational complexity and shifting guest expectations.
The respondent base included leadership and decision-makers from a range of business models and organization sizes, allowing the findings to be interpreted through multiple segmentation lenses rather than relying solely on broad averages.
Qualified Respondents Across Roles and Operating Environments
Sample design varies by study. The examples below illustrate how research populations are constructed around the practitioners and organizations most relevant to the question being examined.
How We Identify Top Performers
“Top-performing” is not a universal label applied the same way in every study. The relevant definition depends on the business initiative and the outcomes being examined.
Where the data support performance tiers, we define the groups using relevant outcomes, such as productivity, reliability or guest experience. We then examine which practices differ between groups. The practices being evaluated do not determine who is classified as a top performer.
What Gets Measured Is Part of the Benchmark
Performance measurement is not an afterthought in the Starfleet Research framework. The metrics organizations choose, how frequently those metrics are reviewed and how results are used to guide improvement can be as important as the technology itself.
Understanding the Context Behind a Study
Starfleet Research produces independently initiated research as well as studies supported by outside underwriters and custom research engagements. Readers should be able to understand the research context and distinguish proprietary survey findings from external market information.
Published studies may include research notes describing the respondent population, qualification criteria, organization types, roles, geographic mix and other sample characteristics relevant to interpreting the findings.
When a published research project receives underwriting support, the supporting organization is identified in the publication. Sponsor and underwriter information is presented separately from the description of the research sample and findings.
Proprietary survey findings should be identified as Starfleet Research findings. External statistics, industry data and third-party research used for context should be attributed to their respective sources.
Sample composition, terminology, outcome measures and analytical approaches can vary by study. Findings should therefore be interpreted within the scope and respondent population described for that research.
See the Methodology in Action
Explore research findings, benchmark studies and decision guidance across the industries and technology markets we cover.