Intermediate Django

Filter QuerySets Based on Advanced Criteria

We now have all of the necessary functionality for creators to share and upload their blog and photo posts. Next, let’s build out a more advanced feed.

The site will have two feeds:

• A blog and photo feed that we will build together.

• A photo-only feed that you will build yourself.

If you have done some work with Django models, you should know how to retrieve a single instance of a model using  get()  and also how to return filtered QuerySets using the  filter()  and  exclude()  methods.

Let's examine more advanced methods of filtering and methods of combining different QuerySets.

Step 1: Create Complex Searches With Field Lookups

Let's build the view that will be our blog and photo feed page. We will do this on the existing homepage, so it’s the first thing someone sees when they log in.

The first task is to retrieve the blog posts of creators to whom the logged-in user has subscribed:

• You can find a QuerySet of  User  objects someone follows with User.follows.all() .

• We want to retrieve all of the  Blog  instances linked to those users by its  contributors  field.

To do this, use field lookups. 

We can use the field lookup in . To use this, you must specify the field,  contributors  , followed by a double underscore,  __  , and then the field lookup,  in  . You then use that as a keyword when filtering. 

In this case, you'll get  contributors__in  . 

Here's what it looks like in the view:

# blog/views.py
def home(request):
    blogs = models.Blog.objects.filter(contributors__in=request.user.follows.all())
    context = {
        'blogs': blogs,
    }
    return render(request, 'blog/home.html', context=context)

Step 2: Use Field Lookups to Query Model Relationships

Next, you want to get photos for the feed. You have two conditions for filtering the QuerySet:

• The  Photo.uploader  field must be a user who is followed.

• You want to exclude photos that are already attached to the fetched  Blog  instances. 

For the first point, use a filter similar to the first one:

photos = models.Photo.objects.filter(
    uploader__in=request.user.follows.all())

For the second point, exclude the photos that are already linked to blog instances.

To do this, stack the  exclude  method onto the same QuerySet as the first. 

Although the  Photo  model does not have an attribute  blog  , you can still query on it because the  Blog  model has a  ForeignKey  to  Photo  . Specify this reverse relationship by querying the lowercase format of the model name, giving you  blog  . 

To exclude these instances, you can use:

blogs = models.Blog.objects.filter(contributors__in=request.user.follows.all())

photos = models.Photo.objects.filter(
    uploader__in=request.user.follows.all()).exclude(
        blog__in=blogs
)

Step 3: Construct OR Lookups With  Q  Objects

The starred  attribute is also on the  Blog  model. Creators can use this to indicate blog posts they would like to display on everyone’s feed, regardless of whether the user follows them or not. 

So far, you have only learned about queries where the lookups can be filtered by being “ANDed” together. You won't get the desired result if you try that here.

You would get:

blogs = models.Blog.objects.filter(contributors__in=request.user.follows.all(), starred=True)

This only returns the blogs for which at least one of the  contributors  is in  user.follows  and the  Blog.starred  is  True  . 

However, we want one of the contributors to be in  user.follows or  if  Blog.starred is  True . 

With  Q  objects. 

You can use Q  objects to hold certain queries that you can then apply in filters. They can be logically combined with AND,  &  ; OR,  |  ; and  NOT  ,  ~  operations. 

These allow you to construct complex database queries beyond what  filter()  and  exclude()  can do alone. 

To query the blogs depending on whether one of the  contributors  is in  user.follows  or if  starred  is  True ,  join two  Q  objects containing your queries with an OR operator,  | , like so:  

from django.db.models import Q

blogs = models.Blog.objects.filter(
    Q(contributors__in=request.user.follows.all()) | Q(starred=True))

You can also negate Q objects by using the NOT operator,  ~  . The query below will return a QuerySet identical to the first:

from django.db.models import Q

blogs = models.Blog.objects.filter(
    Q(contributors__in=request.user.follows.all()) | ~Q(starred=False))

That works, but the first approach looks cleaner.

Step 5: Combine all the Queries in the View

Combine all of these techniques in the view:

from django.db.models import Q

def home(request):
    blogs = models.Blog.objects.filter(
        Q(contributors__in=request.user.follows.all()) | Q(starred=True))
    photos = models.Photo.objects.filter(
        uploader__in=request.user.follows.all()).exclude(
            blog__in=blogs
    )
    context = {
        'blogs': blogs,
        'photos': photos,
    }
    return render(request, 'blog/home.html', context=context)

All of the  Blog  and  Photo  instances are now in the view, but if you want to display them as a single feed, you'll need to combine and order them in some way. Let’s do that next. 

Join and Order QuerySets of Different Model Types

Now that we have retrieved the two QuerySets for the  Photo  and  Blog  models, we need to join them into one list to display them in our feed. Let's see how we can do that.

If you only want to order a QuerySet of one model type, you can use the  order_by()  method, passing it the field that you want to use to order the sequence. You can also prepend (add) a  -  in front of the field to reverse the QuerySet order. 

To retrieve the Blog  instances with the most recent first, you can run: 

blogs = blogs.order_by('-date_created')

This ordering is executed in SQL, so it is faster than loading and ordering Python lists. Unfortunately, you can't do this if you are combining QuerySets of different model types. So, you have to revert to ordering in Python.

There are some things that you can do to improve the performance, though, the key one being using itertools.chain to join the QuerySets. This method returns an iterator that loops through all of the provided iterables as if it were a single sequence of objects. 

You can then feed the result of this to the  sorted  function to return a sorted list. 

Combining these two with a  lambda  function to sort by the  date_created  key gives you: 

from django.db.models import Q

def home(request):
    blogs = models.Blog.objects.filter(
        Q(author__in=request.user.follows) | Q(starred=True))
    photos = models.Photo.objects.filter(
        uploader__in=request.user.follows).exclude(
            blog__in=blogs
    )
    blogs_and_photos = sorted(
        chain(blogs, photos),
        key=lambda instance: instance.date_created,
        reverse=True
    )
    
    
    context = {
        'blogs_and_photos': blogs_and_photos,
    }
    return render(request, 'blog/home.html', context=context)

There is now a list of the  Photo  and  Blog  instances for the feed, sorted with the most recent instances first. Everything is in place to build a combined main feed featuring photo and blog posts! They are even combined into one list in the context to pass to the template.

Next, you are going to retrieve the QuerySets for the photo-only feed.

Exercise: Fetch Posts for the Photo-Only Feed

The view is built for the main feed, but we also want another feed for photos only.

You will need to build a view that will serve this photo-only feed. It will need to:

• Be called  photo_feed  .

• Fetch a QuerySet of  Photo  instances, for which the  uploader  is followed by the user, ordered by most recent. The ordering should be done with the QuerySet API.

• Include the QuerySet in the  context  of the template under the key  photos  .

• Return the rendered template  blog/photo_feed.html  - no need to create this template, we will make it later. 

When you’re ready, check your work against the solution.

Let's Recap!

• You can use double underscores to perform field lookups on models separate to the target model when filtering QuerySets.

• You can use  Q  objects to construct advanced database queries  for retrieving objects using  AND  ,  OR , and  NOT  boolean logic.

• You can sort QuerySets using the  order_by  method.

• You can create an iterator from QuerySets of different model types by using  itertools.chain  and turn them into a sorted list using  sorted  .

• The power of QuerySet goes far beyond the few examples here. Study the official Django documentation for even more useful filters.

Now that you understand the advanced features of QuerySets, you're ready to display your fetched posts in the feed.