Community survey analysis and revised implementation of search in Drupal CMS (formerly Starshot)

As track leads for advanced search in the Drupal CMS product (formerly known as Starshot), we developed a concept and aligned it with other track leads, the leadership team, and the community. We agreed to implement four recipes for Search API, Facets, Search API autocomplete, and Search API decoupled that will be separately installable. This approach allows Drupal CMS to include only the desired recipes by default while keeping others optional.

About the survey

As track leads for advanced search in the Drupal CMS product, we developed a concept and collaborated with other track leads and the leadership team to align on the overarching direction.

To refine this concept further, we conducted a community survey to gather feedback on the proposed default functionality, identify potential gaps, and optimise the implementation strategy.

We are pleased to report that nearly 100 participants took part in the survey, most of whom have substantial experience and provided valuable insights and suggestions. 

This comprehensive feedback has been instrumental in shaping a refined concept, highlighting key areas for consideration and ensuring that the Drupal CMS Advanced Search aligns closely with the broadest range of real user needs.

Below, we present these findings and the resulting implementation plan.

Survey results

Summary and conclusions

The survey reveals a strong preference of almost 90% for Search API as the standard search technology in Drupal CMS. Similarly the related technologies Facets (88%), Search API autocomplete (87%), and Search API decoupled (78%) were also recommended for inclusion.

However, the participants expressed concerns about complexity, performance, and suitability for different user groups. Participants generally favoured more flexible, configurable solutions that can cater to a wide range of use cases and experience levels.

See survey results in a Google sheet

Participant overview

A total of 99 participants took part in our survey, sharing valuable information about their backgrounds and expertise. Here's a breakdown of their proficiency levels, years of experience, and areas of expertise:

Proficiency levels:

• Experts: The majority of participants, 65 in total, identified themselves as experts in their respective fields.
• Intermediate: 28 participants categorised themselves as having an intermediate level of proficiency.
• Beginner: 6 participants indicated they were beginners.

Years of experience:

• More than 5 years: 77 participants have over five years of experience, showcasing a highly experienced group.
• 2-5 years: 14 participants reported having between two to five years of experience.
• Less than 2 years: 8 participants are relatively new to their fields, with less than two years of experience.

Areas of expertise: 

Participants were allowed to select multiple areas of expertise. Here’s how they identified their specialties:

• Developers: 82 participants, including both frontend and backend developers.
• Frontend: 16 participants specified their expertise in frontend development.
• Backend: 64 participants specialised in backend development.
• Non-Developers: 17 participants included roles such as DevOps, site builders, editors, and marketers.
• Decision Makers: 13 participants identified as decision-makers within their organisations.
• Strategists: 20 participants described their roles as strategists.

This diverse mix of participants, varying in experience and specialties, provides a well-rounded perspective for our survey insights.

Survey results

The survey explored opinions on various search-related functionalities for Drupal CMS, including the use of "Search API" versus "Core Search", search facets, autocomplete, database search backend, and decoupled search. Below is a summary of the results and concerns raised by participants.

Results

86 participants (87%) prefer using Search API, while 13 prefer Core Search.

Considerations for using Search API:

• Search API has a high learning curve, especially for less experienced developers and no-code site builders, which could be mitigated by improved documentation or more intuitive configuration interfaces.
• Some participants raised concerns about the need for better control over what gets indexed and how search results are configured.
• Potential performance issues were highlighted, especially with large nodes and complex rendered content in search results, emphasising the need for clearer guidance and messaging.

Results

87 (88%) participants support including Search Facets.

Considerations for including Facets:

• Facets add complexity and may not be needed for all use cases, especially for less experienced users.
• The Facets module is still evolving, and there are concerns about its stability and integration, suggesting that it may be better to wait for a more stable version.
• Some participants suggested that facets should be configurable and optionally enabled based on specific content types or fields.

Results

86 (87%) participants support including Search Autocomplete.

Considerations for including autocomplete:

• Some participants believe autocomplete could mislead users into selecting unintended search terms rather than performing a full search.
• It may not be necessary for most sites and could be considered an advanced feature.

Results

86 participants (87%) support using the database as a search backend.

Considerations on using the database backend:

• There are performance concerns, particularly with increased database load, suggesting that using dedicated search backends like Solr, Elasticsearch, or noSQL solutions may offer better alternatives.
• The database search is seen as a basic solution that may not scale well for larger or more complex sites.

Should “Decoupled Search” be included?

Results

 77 (78%) participants support including Search API decoupled.

Considerations for including decoupled search:

• This functionality may not be relevant for the primary target audience (marketers and no-code developers) and adds unnecessary complexity.
• Decoupled search might be better suited as an optional, separate recipe for more advanced users who need it.

Refined implementation of default functionality (“Recipes”)

The community survey results confirmed that search functionality in Drupal CMS is highly project-dependent. As a result, the default functionality should be provided though separately installable recipes, allowing users to include only the necessary features and optionally add more as required.

After analysing the survey feedback and collaborating with other track leads and the community, we have defined the following recipes to be included in the standard configuration of Drupal CMS:

• Search API Base Recipe: Utilises the database backend for basic search functionality.
• Facet Recipe: Includes search facets based on content type bundles.
• Search API Autocomplete Recipe: Offers a standard configuration for search autocomplete.
• Search API Views Recipe: Provides a standard configuration for traditional (coupled) search setups
• Search API Decoupled Recipe: Provides a standard configuration for decoupled search setups.

These recipes ensure flexibility and customisation, catering to a wide range of use cases and user needs.

Thank you to all participants

This community survey embodies what is great about Drupal and Open Source approaches to development. The benefit of many eyes and collaborative contribution leads to better solutions. We highly appreciate you taking the time to share your knowledge and insights.

How to get involved with Drupal CMS Search Track

Are you inspired to get more involved and contribute to Drupal CMS or the Advanced Search Track? There are a multitude of ways to help bring Dries' ambitious vision for Drupal CMS to life. Head over to the Starshot Initiative to find out more and get involved. You can also stay updated on the Drupal CMS Advanced Search by subscribing to our monthly newsletter.