Snowflake Universal Search 101—Quickly Locate Your Data (2024)

Manually searching for data in Snowflake often feels like hunting for a needle in a haystack—incredibly frustrating and time-consuming. Snowflake recognizes this difficulty and has created Universal Search to completely change the way you search for your data. Built on top of the powerful Snowflake Cortex, Universal Search is designed to make finding data as intuitive as a simple search. Currently in public preview, this LLM-powered search leverages advanced AI models and search engine technology from Neeva, which Snowflake acquired in May 2023. It allows users to effortlessly locate various Snowflake artifacts, like tables, views, databases, schemas, data products from the Snowflake Marketplace, relevant Snowflake documentation topics, and Snowflake Community Knowledge Base articles—all within the Snowflake environment.

In this article, we will cover everything you need to know about Snowflake Universal Search—how it works, its key features, best practices, a step-by-step guide to navigating and utilizing it, as well as the considerations and limitations of Snowflake Universal Search.

What is Snowflake Universal Search?

Snowflake Universal Search is a comprehensive search solution designed to enhance data discoverability and accessibility within the Snowflake ecosystem. Its primary purpose is to help users with the ability to quickly and efficiently locate Snowflake artifacts like Dashboards, Worksheets, Databases, Schemas, Tables, Views, User-defined functions (UDFs), Stored procedures, data products from the Snowflake Marketplace, Snowflake documentation, and relevant Snowflake Community Knowledge Base articles.

Snowflake Universal Search's major goal is to simplify data discovery by allowing users to conduct natural language searches. This means that you may search for anything by typing it in normal, natural English. This feature allows users to easily find information, even if their search terms are misspelled or incomplete.

Here is a TL;DR version of Snowflake Universal Search's core capabilities:

• Object Search: Locate tables, views, schemas, and databases.
• Snowflake Marketplace Search:  Find and access data products available for purchase or use.
• Documentation Search: Access documentation articles related to your queries.
• Community Knowledge Search: Get community-contributed knowledge articles relevant to your search terms.

Technical Breakdown of Snowflake Universal Search—Features and Integration

Now, in this section, we will explain the technical integration, functionalities, and user interface features that make Snowflake Universal Search an effective tool for data discovery.

Integration with Snowflake Cortex

Snowflake Universal Search is integrated with Snowflake Cortex, leveraging its advanced machine learning (ML) and natural language processing (NLP) capabilities. This integration enhances the accuracy and relevance of search results by utilizing vector search functionality and large language models (LLMs). Snowflake Cortex provides the necessary infrastructure for running these ML models, allowing Snowflake Universal Search to handle complex queries efficiently.

Search Capabilities

Snowflake Universal Search can search a wide range of items, including database objects, data products, documentation, UDFs, and stored procedures. This broad search capability gurantees users can find various types of information within a single query.

Natural Language Processing

Snowflake Universal Search uses NLP capabilities to understand and process natural language queries. Users can input search terms as they would in a conversation, making the search process more intuitive and user-friendly. This capability is powered by LLMs that parse and interpret the intent behind user queries, providing highly relevant results.

Intelligent Query Understanding

Snowflake Universal Search handles misspellings and partial terms intelligently. This means that even if a user inputs an incomplete or incorrectly spelled term, the search engine can still provide relevant results.

Organized Search Results

Search results are categorized by type, helping users quickly find the specific information they need. Categories include:

• Tables & Views: Results related to tables and views.
• Databases & Schemas: Results related to databases and schemas.
• Functions & Procedures: Results related to user-defined functions and stored procedures.
• Marketplace: Results related to data products and other marketplace items.
• Documentation: Results related to documentation and help resources.

Metadata Search

Snowflake Universal Search indexes metadata such as:

• Names: Object names and column names.
• Comments: Comments associated with objects and columns.
• Tags: Tags associated with objects and columns.

However, it does not search within the contents of the database objects themselves, focusing instead on the metadata.

Embedding-Based Ranking

The search algorithm of Snowflake Universal Search uses embeddings—generated by a bi-encoder model to convert table names and other database objects into numerical representations. These embeddings are used to semantically rank results relative to a text query, making sure that the most relevant results are displayed at the top. The system is trained on a diverse dataset to handle various query types, including head queries (simple, one-word) and tail queries (complex, natural language).

Step-by-Step Guide to Navigate and Utilize Snowflake Universal Search

Let's dive into a detailed walkthrough on how to effectively use Snowflake Universal Search, from signing in to Snowsight to accessing and interpreting search results.

Step 1—Sign In to Snowsight

To access Snowflake Universal Search, users must first sign in to Snowflake. Once signed in, you will be able to access Snowsight, the web UI of Snowflake. It provides a centralized platform for data exploration, analysis, and collaboration within the Snowflake ecosystem.

Step 2—Accessing the Search Feature

Once logged in to Snowsight, you can access the Snowflake Universal Search feature by clicking on the "Search" option in the navigation menu.

This will open the Snowflake Universal Search interface, where you can enter your own search queries.

Step 3—Enter Search Terms

Now, in the Snowflake Universal Search input field, you can enter your own search terms. As mentioned earlier, one of the key strengths of Snowflake Universal Search is its ability to understand natural language queries. You can input the search terms using conversational language.

Tips and Best Practices for Entering Search Terms

1) Use natural language to describe your search intent

2) Experiment with different variations of search terms

3) Include relevant keywords or phrases related to your search topic

4) Leverage metadata information, such as object descriptions or tags, in your search queries

5) Don't worry about exact spellings; Snowflake Universal Search can handle variations and partial terms

Step 4—Review Search Results

After entering your search terms, Snowflake Universal Search will process the query and present the results in a categorized format. The search results will be organized into different categories, such as database objects, data products, documentation topics, and community knowledge base articles.

Each category will display a list of relevant results, providing you with a quick overview of the available data assets and resources related to your search query. You can easily navigate between categories and explore the various types of results.

Step 5—View Details of a Search Result

To gain more insights into a specific search result, click on each individual category to view additional details. Snowflake Universal Search displays all the necessary metadata information to navigate through the details of the search results.

For example:

1) To view only tables or views, select the "Tables & Views" category.

2) To view or open a worksheet or dashboard, select the "Worksheets & Dashboards" category.

3) To view a database or schema, select the "Databases & Schemas" category.

4) For user-defined functions or procedures, select the "Functions & Procedures" category.

As you can see, there are no results in the “Functions & Procedures” category, which means the search input you provided didn't match any entries. To resolve this, you can reset the search and try again.

5) To view all the data products available in Snowflake Marketplace, select the "Marketplace" category.

We will dive into more depth in Step 6.

6) For all documentation, select the "Documentation" category.

7) Also, there is an "All" category if you want to view all the Snowflake artifacts at once.

Note: If a category isn’t displayed, it means there are no results for that category or your active role does not have access to those results

Step 6—Access Marketplace Listings

If the search results include data products available on the Snowflake Marketplace, you can easily access and explore these listings directly from the Snowflake Universal Search interface. By clicking on a Marketplace listing, you can review detailed information about the data product, including its description, pricing, and terms of use.

Note: To access or purchase data products from the Snowflake Marketplace, you must have agreed to the Snowflake Provider and Consumer Terms and meet any additional legal requirements specified for the specific listing.

Scope of Search Results of Snowflake Universal Search

Snowflake Universal Search encompasses a wide range of objects and resources within the Snowflake ecosystem. The following types of objects and resources are indexed and searchable through Snowflake Universal Search:

• Dashboards
• Worksheets
• Databases
• Schemas
• Tables
• Views
• User-defined functions (UDFs)
• Stored procedures
• Data products offered on the Snowflake Marketplace
• Documentation pages from the Snowflake documentation website
• Knowledge base articles from the Snowflake Community

This broad scope makes sure that users can easily find and access a wide range of data assets, resources, and information, all tailored to their specific needs and use cases.

Limitations of Snowflake Universal Search

Snowflake Universal Search is a powerful and capable feature, but before you use it, you should understand its limitations.

1) Results based on active role privileges

Whenever you are searching for database objects within Snowsight, Snowflake Universal Search will only display results that the user's currently active role has privileges to view. So that users can only discover and access data assets they are authorized to work with.

2) Delays for New or Recreated Objects

Newly created database objects or objects that have been dropped and recreated (e.g., by scheduled tasks or automated pipelines) may take up to a few hours to appear in the search results. This delay is due to the indexing process that occurs in the background to ensure the search index remains up-to-date.

3) Supports only English

Currently, Snowflake Universal Search is optimized for search terms and queries in the English language. This limitation may challenge users operating in non-English languages.

4) Limited Regional availability

Snowflake Universal Search is available to accounts in a limited set of Snowflake regions:

• AWS EU (Zurich)
• GCP Europe West2 (London)
• GCP Europe West4 (Netherlands)
• GCP US Central1 (Iowa)
• GCP US East4 (N. Virginia)

Accounts in other regions may not have access to this feature until it is more widely rolled out.

5) Not available in VPS and government cloud regions

Snowflake Universal Search is not available for accounts hosted in Virtual Private Snowflake (VPS) or government cloud regions. These environments have specific security and compliance requirements that may necessitate additional considerations before Universal Search can be made available.

Note: Snowflake is actively working on addressing these limitations and expanding the capabilities of Universal Search. As the feature develops and more people start using it, users can expect to see many improvements.

Further Reading

• Search Snowflake objects and resources
• Use AI in Seconds with Snowflake Cortex
• A peek inside how Snowflake’s new Universal Search feature was built
• Snowflake Provider and Consumer Terms of Service
• Universal Search Release Notes
  

Conclusion

And that's a wrap! Snowflake Universal Search is a powerful tool that makes data discoverability and accessibility a breeze—all within the Snowflake platform. It uses powerful NLP and integrates with Snowflake Cortex to give accurate and relevant search results without the need for third-party applications. As it continues to evolve, Snowflake Universal Search will become an essential tool for simplifying data search and analysis efficiently and conveniently.

In this article, we have covered:

• What is Snowflake Universal Search?
• How Does Snowflake Universal Search Works?
• Step-by-Step Guide to Navigate and Utilize Snowflake Universal Search
• Scope of Search Results of Snowflake Universal Search
• Limitations of Snowflake Universal Search

…and so much more!

FAQs

What is Snowflake Universal Search?

Snowflake Universal Search is a tool that enables users to find database objects, data products, documentation, and knowledge base articles using natural language queries.

Is Snowflake Universal Search available to all Snowflake customers?

Currently, Snowflake Universal Search is in public preview and is initially available to accounts in a limited set of Snowflake regions.

How does Snowflake Universal Search handle misspellings and partial terms?

Snowflake Universal Search uses advanced NLP techniques to understand and process queries, providing relevant results even with misspellings or incomplete search terms.

What types of objects and resources can Snowflake Universal Search search for?

Snowflake Universal Search can search for dashboards, worksheets, databases, schemas, tables, views, user-defined functions (UDFs), stored procedures, data products from the Snowflake Marketplace, Snowflake documentation pages, and Snowflake Community knowledge base articles.

Can Snowflake Universal Search handle misspellings and partial terms?

Yes, Snowflake Universal Search can handle misspellings and partial terms intelligently, providing relevant results even if a user inputs an incomplete or incorrectly spelled term.

Are there any limitations to Snowflake Universal Search?

Yes, search results are based on the active role's privileges, there may be delays in indexing new or recreated objects, it supports only English search terms, and it is available in select Snowflake regions.

What metadata does Snowflake Universal Search index?

Snowflake Universal Search indexes metadata such as names, comments, and tags associated with database objects and columns, but it does not search within the contents of the database objects themselves.

Can I use Universal Search to search within the actual data contents of my database objects?

No, Snowflake Universal Search indexes and searches through metadata associated with database objects and columns, such as names, comments, and tags. It does not search within the actual data contents of the objects themselves.

How frequently is the Snowflake Universal Search index updated with new or modified objects?

Snowflake Universal Search index is updated periodically, and newly created or modified objects may take up to a few hours to appear in the search results.

Does Snowflake Universal Search support natural language queries in languages other than English?

At the moment, Snowflake Universal Search is optimized for natural language queries in the English language.

How do I access Snowflake Universal Search?

Sign in to Snowsight, select "Search" from the navigation menu, and enter your search terms to begin using Snowflake Universal Search.

Can I access and explore data products from the Snowflake Marketplace through Snowflake Universal Search?

Yes, if the search results include data products available on the Snowflake Marketplace, users can access and explore these listings directly from the Snowflake Universal Search interface.

Are search results in Snowflake Universal Search ranked based on relevance?

Yes, Snowflake Universal Search uses embedding-based ranking to semantically rank results relative to a text query, ensuring that the most relevant results are displayed at the top.

Can I search for Snowflake documentation topics using Snowflake Universal Search?

Yes, Snowflake Universal Search can search for and display relevant Snowflake documentation topics related to a user's search query.

Is Snowflake Universal Search available in all Snowflake regions?

No, Snowflake Universal Search is currently available in a limited set of Snowflake regions, including AWS EU (Zurich), GCP Europe West2 (London), GCP Europe West4 (Netherlands), GCP US Central1 (Iowa), and GCP US East4 (N. Virginia).

Can I access Snowflake Universal Search if my account is hosted in a Virtual Private Snowflake (VPS) or government cloud region?

No, Snowflake Universal Search is not available for accounts hosted in Virtual Private Snowflake (VPS) or government cloud regions.