Demystifying Strapi's Populate & Filtering

Introduction

Hello everyone! This discussion is based on Richard's Strapi Conf talk on demystifying Strapi's populated filtering. You can find the whole video here

This blog post serves as a quick refresher on effectively consuming data in Strapi using field selection, population, and filtering.

Why is This Discussion Necessary?

Over the years, Strapi has evolved significantly, and the most noticeable change has been in how we interact with our content. Content consumption is a crucial part of the data interaction experience. This post aims to help you become comfortable with the tools you need to interact with your data efficiently.

You can learn about these topics in our documentation. Check out the following sections, populate and filtering, as well as, this video where we discuss the topic in more details.

Getting Started with Population and Field Selection

Let's dive right into the concepts of population and field selection.

As you can see above, you can only get in if your name is on the list. Strapi operates this way with population and field selection.

By default, Strapi does not automatically populate or select your relations for you – you must do it manually.

This approach saves bandwidth from the database queries Strapi generates. As you interact with your data, for instance, through the REST API, you need to enable permissions for specific operations.

For instance, if you have a user with an articles (a relation), you can't populate this relation unless you enable the find permission on the collection type, as shown below.

So here are some things to consider when working with populate and filtering in Strapi.

Note: Remember the depth of your population and selection, as complex requests can take longer to execute.

Exploring Population and Field Selection with Examples

Let's go through some examples. I have created a repl environment that uses the qs library to generate LHS Bracket syntax strings that we can use to test out our queries in the browser or using Insomnia.

You can find the repl here with a basic example.

But for simplicity, I will be using Strapi's Query Builder tool, which you can find here.

Here is a helpful Chrome extension to make your JSON more beautiful when viewing in the browser. JSON Viewer

In my Strapi application, I have created several collection types and relations, which I will explain as we proceed. We have top-level relations, top-level fields, dynamic zones, and components within those dynamic zones.

Let's take a look at some examples of population and field selection.

Populate Everything with the Wildcard Operator

Before looking at the wildcard * operator, let's look at what we get as a response when querying our articles without passing and populating or filtering options.

You can run this GET query in Insomnia, but we will just look at it in the browser for brevity.

You can download this chrome extension to make your JSON readable.

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles

In our response below, we can see that we only return the top-level fields and nothing else. None of the other relations, components, or dynamic zones.

If you want to bring back as much information as possible, use the wildcard * operator.

1{
2  populate: "*",
3}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate=*

Remember that some details may not be visible, such as avatar and articles within the authorsBio relation, due to limited population depth.

When using the wildcard * operator, you will only populate relations, dynamic zones, and components to a depth of one.

You can learn more in our documentation on the wildcard operator here.

Below, we have additional relations avatar and articles; let's look to see how we can populate those fields next.

Populate Specific Relations

To populate specific relations, we must specify which fields we want to populate in Strapi. Let's take a look at the following example.

Instead of using the wildcard, we will build out the query manually. Here is what our current query looks like without any options being passed.

Let's add this populate query. We can use this array populate notation to specify which fields we want to populate. The following will get the same data as when we used the wildcard and populated one level deep for all the specified items.

1{
2  populate: [
3    "cover",
4    "category",
5    "blocks",
6    "authorsBio",
7    "seo"
8  ],
9}

or using the object notation

1{
2  populate: {
3    cover: {
4      populate: true,
5    },
6    category: {
7      populate: true,
8    },
9    blocks: {
10      populate: true,
11    },
12    authorsBio: {
13      populate: true
14    },
15    seo: {
16      populate: true
17    }
18  },
19}

In most complex queries, object notation is preferred. We will see why later in the post.

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[0]=cover&populate[1]=category&populate[2]=blocks&populate[3]=authorsBio&populate[4]=seo

Before looking at how we can populate nested relations, let's look at how we can populate specific fields.

Populate Nested Relations

Let's look at how we can populate our nested relationships. We will use the above query as a starting point.

We will look at how to populate additional relations in our authorsBio relation and seo component.

1{
2  populate: [
3    "cover",
4    "category",
5    "blocks",
6    "authorsBio.avatar",
7    "authorsBio.articles",
8    "seo.shareImage"
9  ],
10}

or using the object notation

1{
2  populate: {
3    cover: {
4      populate: true,
5    },
6    category: {
7      populate: true,
8    },
9    blocks: {
10      populate: true,
11    },
12    authorsBio: {
13      populate: ["avatar", "articles"]
14    },
15    seo: {
16      populate: ["shareImage"]
17    }
18  },
19}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[0]=cover&populate[1]=category&populate[2]=blocks&populate[3]=authorsBio.avatar&populate[4]=authorsBio.articles&populate[5]=seo.shareImage

In the response below, we can see the populated nested relationships.

Now, let's look at the avatar response within authorsBio looks like.

There are a lot of fields. What if we wanted to populate selected fields only. That is exactly what I will show you in the next section.

Populate and Field Select

We will now see how you can tell Strapi only to return specific fields. We will also see where the object notation is useful since array notation does not allow you to do this.

Array notation example:

1{
2  populate: [
3    "cover",
4    "category",
5    "blocks",
6    "authorsBio.avatar",
7    "authorsBio.articles",
8    "seo.shareImage"
9  ],
10}

Object notation example:

1{
2  populate: {
3    cover: {
4      populate: true,
5    },
6    category: {
7      populate: true,
8    },
9    blocks: {
10      populate: true,
11    },
12    authorsBio: {
13      populate: ["avatar", "articles"]
14    },
15    seo: {
16      populate: ["shareImage"]
17    }
18  },
19}

Let's refactor our object notation example to see how to populate only specific fields. We will focus on the authorsBio avatar field.

This is what our original response looked like.

Let's now only return the name, alternativeText, caption, and url.

To accomplish this, let's make the following changes to our query.

1{
2  populate: {
3    cover: {
4      populate: true,
5    },
6    category: {
7      populate: true,
8    },
9    blocks: {
10      populate: true,
11    },
12    authorsBio: {
13      populate: {
14        avatar: {
15          fields: [
16            "name",
17            "alternativeText",
18            "caption",
19            "url"
20          ]
21        },
22        articles: {
23          populate: true,
24  }
25      }
26    },
27    seo: {
28      populate: ["shareImage"]
29    }
30  },
31}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[cover][populate]=true&populate[category][populate]=true&populate[blocks][populate]=true&populate[authorsBio][populate][avatar][fields][0]=name&populate[authorsBio][populate][avatar][fields][1]=alternativeText&populate[authorsBio][populate][avatar][fields][2]=caption&populate[authorsBio][populate][avatar][fields][3]=url&populate[authorsBio][populate][articles][populate]=true&populate[seo][populate][0]=shareImage

Notice in the response below we returned all of our data from before and only the specified fields for our avatar.

In the next section, we will look at complex population and **field selection.

Complex Population and Field Selection

In this example, instead of getting our articles, we will look to populate our home page. We will look first find our page by slug. We will populate our dynamic zone and look only to get the data for our testimonials group component.

We will build this query in steps.

First, let's find our page via slug. We will use filters to filter where the slug is equal to home. We will look at filters in more detail in the next section.

1{
2  filters: {
3    slug: "home";
4  }
5}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home

We get the following response.

We can populate our dynamic zones with what we already learned. Let's modify our query with the following.

1{
2  filters: {
3    slug: "home"
4  },
5  populate: {
6    contentSections: {
7      populate: true,
8    }
9  }
10}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home&populate[contentSections][populate]=true

The above response shows all of the contentSections items populated to the first depth.

From what we learned above, we can add to our query to populate nested relations and fields.

But instead, I want to show you how to populate individual dynamic zones, components, and their fields using the on option.

You can see our docs for additional information here.

Let's make a query to populate the following dynamic zone.

We will focus on only returning the testimonials-group component from our dynamic zone with the title field and the nested testimonials component with the text and authorName fields.

Let's make the following changes.

1{
2  filters: {
3    slug: "home"
4  },
5  populate: {
6    contentSections: {
7      on: {
8      "sections.testimonials-group": {
9        fields: ["title"],
10        populate: {
11        testimonials: {
12          fields: ["text", "authorName"]
13        }
14        }
15      }
16      }
17    }
18  }
19}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home&populate[contentSections][on][sections.testimonials-group][fields][0]=title&populate[contentSections][on][sections.testimonials-group][populate][testimonials][fiedls][0]=text&populate[contentSections][on][sections.testimonials-group][populate][testimonials][fiedls][1]=authorName

As you can see in the response, we only return the testimonials-group component from the dynamic zone with the selected fields and the nested testimonials component with the specified fields.

Now that we know how to populate individual components from our dynamic zones, we will look at how to apply filtering to our query in the next section.

Filtering Data

Filtering allows you to retrieve specific information from your application. However, note that deep filtering is not available in dynamic zones, and complex filters may cause performance issues. You can learn more about filtering here.

Here are some filtering examples:

Top-level Filtering Example

We already saw this example when we filtered our pages by slug.

1{
2  filters: {
3    slug: "home";
4  }
5}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home

By default, the above notation uses the $eq operator; we can also write this with the following.

1{
2  filters: {
3    slug: {
4      '$eq': "home"
5    }
6  }
7}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug][$eq]=home

Here are some examples of common operators.

You can see the complete list here

Let's look for a page with the shortName field set to null.

We can run the following query.

1{
2  filters: {
3    shortName: {
4      "$null": true
5    }
6  }
7}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[shortName][$null]=true

Filtering by Relation Example

You can filter based on relations, like listing all the articles written by a specific author. We will filter our articles based on the name in the authorsBio.

1{
2  populate: {
3   authorsBio: {
4     fields: ["id", "name", "email"]
5   }
6  },
7  filters: {
8    authorsBio: {
9      name: {
10      "$eq": "Megan"
11      },
12    }
13  }
14}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[authorsBio][fields][0]=id&populate[authorsBio][fields][1]=name&populate[authorsBio][fields][2]=email&filters[authorsBio][name][$eq]=Megan

In the response below, we can see all the articles Megan wrote.

Complex Filtering

You can use complex filters, like combining multiple conditions using the "and" operator. Let's refactor the above query to see this in action.

In the following query, we would only like to return articles by Megan and a category named Strapi using our $and operator. Notice how it takes an array as an argument.

1{
2  fields: ["title"],
3  populate: {
4   authorsBio: {
5     fields: ["name"]
6   },
7   category: {
8     fields: ["name"]
9   },
10  },
11  filters: {
12    $and:
13      [ { authorsBio: {  name: { "$eq": "Megan" } } }, { category: { name: { "$eq": "strapi" } } }],
14    }
15}

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?fields[0]=title&populate[authorsBio][fields][0]=name&populate[category][fields][0]=name&filters[$and][0][authorsBio][name][$eq]=Megan&filters[$and][1][category][name][$eq]=strapi

In the following response, you can see that we only return posts that were written by Meagan and belong to Strapi category.

1{
2  "data": [
3    {
4      "id": 1,
5      "attributes": {
6        "title": "The Benefits of a Headless CMS Like Strapi",
7        "authorsBio": {
8          "data": { "id": 2, "attributes": { "name": "Megan" } }
9        },
10        "category": { "data": { "id": 1, "attributes": { "name": "strapi" } } }
11      }
12    },
13    {
14      "id": 2,
15      "attributes": {
16        "title": "Unleashing the Power of Customization with Strapi CMS",
17        "authorsBio": {
18          "data": { "id": 2, "attributes": { "name": "Megan" } }
19        },
20        "category": { "data": { "id": 1, "attributes": { "name": "strapi" } } }
21      }
22    }
23  ],
24  "meta": {
25    "pagination": { "page": 1, "pageSize": 25, "pageCount": 1, "total": 2 }
26  }
27}

Conclusion

To conclude, today, we demystified populate and filtering in Strapi.

We strongly recommend that you are the one who writes the populate and filtering logic in your app and not rely on external plugins such as Populate Deep.

Yes, it is an easy solution, but it comes at the cost of performance since it will populate all the data to the depth specified.

From what we covered above, Strapi's populate and filtering gives you granular control of what data you want to return from your api.

This allows you to only get the data that you need. You can also add the populate and filtering inside a route middleware, allowing you to write less code on your front end.

Here is an excellent article written by Kellen Bolger that you should check out here.

Hope you enjoyed this article. The API we have been using to test our queries is hosted on Strapi Cloud. You can learn more about it here.

As always, thank you for stopping by, and remember to join the Strapi community on Discord to discuss and learn more.

We have daily open office hours Monday - Friday from 12:30 PM CST to 1:30 PM CST.

Thanks for reading!