Hugo Soltys
Hugo Soltys

My name is Hugo, I'm 27 and I'm a Symfony developer since 2013. I love to create websites by myself to learn new technologies or increase my skills.

iGraal
2019-11-21 - Today Freelance Symfony developer
M6 Web
2018-10-16 - 2019-11-20 Symfony developer
Decathlon
2016-01-01 - 2018-10-15 Symfony developer
IT Room
2014-09-01 - 2015-12-31 Symfony developer
Noogaa
2013-09-01 - 2014-08-31 Student symfony developer
Work with me

Autocomplete search with Elasticsearch and Symfony 4

June 14, 2019 by Hugo - 3854 views - 0 Comments


Autocomplete search with Elasticsearch and Symfony 4

In this article, we will learn how to implement an autocomplete search bar in your Symfony application with Elasticsearch step by step.


For the following, I will assume that you have a running Elasticsearch on your server. You can find all the installation instructions on the official documentation.

In this article, we will assume that you manage a superhero database and you want to index them in an Elasticsearch index so you can search them with a powerful autocomplete input.

1. Installing FOSElasticaBundle

FOSElasticaBundle is an Elasticsearch PHP integration for Synfony using Elastica. Still the easiest way to use Elasticsearch with Symfony nowadays.

Open a terminal and type :

$ composer require friendsofsymfony/elastica-bundle

 

2. Configuring the bundle

Add your Elasticsearch server host and port in your .env and .env.dist files.

ELASTICSEARCH_HOST=localhost
ELASTICSEARCH_PORT=9200

We will create a marvel index containing the superhero type. To do so, open your config/packages/fos_elastica.yaml (this file should be created by a flex recipe) and configure it like the following.

fos_elastica:
    clients:
        default: { host: '%env(resolve:ELASTICSEARCH_HOST)%', port: '%env(resolve:ELASTICSEARCH_PORT)%' }
    indexes:
        marvel: # the name of our index
            settings:
                index:
                    analysis:
                        analyzer:
                            keyword_analyzer: # this is a custom analyzer, see the explanations below
                                type: custom
                                tokenizer: standard
                                filter: [standard, lowercase, asciifolding, trim]
            types:
                superhero: # the name of our type
                    properties:
                        name:
                            analyzer: keyword_analyzer
                            search_analyzer: keyword_analyzer
                            type: text
                    persistence:
                        driver: orm
                        model: App\Entity\SuperHero
                        provider: ~
                        finder: ~
                        repository: App\SearchRepository\SuperHeroRepository

 

Let's take a look to understand what we did.

  • fos_elastica.clients.default : the host and the port where your elasticsearch instance is running
  • keyword_analyzer : a custom analyzer that will tell to Elasticsearch how it must store your tokens in his index
    • standard tokenizer : divides text into words removing most punctuation symbols (eg. Captain Marvel will be stored as Captain; Marvel;)
    • lowercase filter : will store your tokens in lowercase (Captain; Marvel; becomes captain; marvel;)
    • asciifolding filter : will convert all the characters which are not in the first 127 ASCII characters into their ASCII equivalents
    • trim filter : will trim the whitespaces surrounding a token
  • superhero.properties : list all the properties you want to store in your index
    • analyzer and search_analyzer : we specify here that we will use our custom analyzer. It will ensure that your query terms are in the same format as the inverted index
  • persistence : in this article, the Elasticsearch index is based on Doctrine, so you have to specify the concerned entity and, if you want, the path to your dedicated search repository

3. Writing the entity

FOSElasticaBundle ships with support for Doctrine objects, so we will need a SuperHero entity like specified in the above configuration.

To make it simple, we will just add id and name properties.

<?php

namespace App\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity()
 */
class SuperHero
{
    /**
     * @ORM\Id()
     * @ORM\GeneratedValue()
     * @ORM\Column(type="integer")
     */
    private $id;

    /**
     * @ORM\Column(type="string", length=255)
     */
    private $name;

    public function getId(): ?int
    {
        return $this->id;
    }

    public function getName(): ?string
    {
        return $this->name;
    }

    public function setName(string $name): self
    {
        $this->name = $name;

        return $this;
    }
}

 

4. Populate your index

At this point, you can update your database, add some superheroes in your table and finally populate your Elasticsearch index.

$ bin/console doctrine:schema:update --force
$ bin/console doctrine:query:sql "INSERT INTO superhero (id, name) VALUES (1, 'Iron Man'), (2, 'Black Widow'), (3, 'Hulk'), (4, 'Thor'), (5, 'Captain America')"
$ bin/console fos:elastica:populate

 

So now your data looks like this.

ID Name in your database Elasticsearch tokens
1 Iron Man "iron";"man";
2 Black Widow "black";"widow";
3 Hulk "hulk";
4 Thor "thor";
5 Captain America "captain";"america";

 

5. Creating your search repository

Now we have superheroes in our marvel index, we can write a search function that will give us the matching heroes for a given name.

The result of this function will be an array of SuperHero objects which the name property starts with your search.

<?php

namespace App\SearchRepository;

use Elastica\Query;
use Elastica\Query\BoolQuery;
use FOS\ElasticaBundle\Repository;

class SuperHeroRepository extends Repository
{
    public function search($search = null, $limit = 10)
    {
        $query = new Query();

        $boolQuery = new BoolQuery();

        if (!\is_null($search)) {
            $fieldQuery = new Query\MatchPhrasePrefix();
            $fieldQuery->setField('name', $search);

            $boolQuery->addMust($fieldQuery);
        }

        $query->setQuery($boolQuery);
        $query->setSize($limit);

        return $this->find($query);
    }
}

 

Be careful, the MatchPhrasePrefix query is only available for Elasticsearch 5.0 or higher.

6. Writing the controller

Alright. At this point, we have the data and a function that allow us to search in.

Now, we need a controller which will actually do the search when it will be called.

<?php

namespace App\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;
use FOS\ElasticaBundle\Manager\RepositoryManagerInterface;
use App\Entity\SuperHero;
use App\SearchRepository\SuperHeroRepository;

class SuperHeroController extends AbstractController
{
    /**
     * @Route("/superhero/search", name="superhero_search")
     *
     * @param RepositoryManagerInterface $manager
     * @param Request $request
     *
     * @return JsonResponse
     */
    public function searchSuperHero(RepositoryManagerInterface $manager, Request $request)
    {
        $query = $request->query->all();
        $search = isset($query['q']) && !empty($query['q']) ? $query['q'] : null;

        /** @var SuperHeroRepository $repository */
        $repository = $manager->getRepository(SuperHero::class);

        $superheroes = $repository->search($search);

        /** @var SuperHero $superhero */
        foreach ($superheroes as $superhero) {
            $data[] = [
                'name' => $superhero->getName(),
            ];
        }

        return new JsonResponse($data);
    }
}

 

7. Creating the template

Now our backend is ready, let's create a template with our form search input.

<form>
    <label for="q">Search a superhero</label>
    <input class="form-control"
        type="text"
        name="q"
        id="search-input"
        data-suggest="{{ path('superhero_search') }}"
    >
</form>

 

8. Your favorite part, the Javascript

To make your input automplete automatically as long as you are typing, I will show you a way to do so with a JS library called bootstrap-typeahead.

I know this is probably not the best way to do since this library is a bit old. Feel free to suggest a better solution in the comments :)

You can install it by running the following command in your terminal.

# with Yarn
$ yarn add bootstrap-typeahead

# with npm
$ npm i bootstrap-typeahead

 

Once you installed it, open your main js file and add the following.

require('bootstrap-typeahead');

$(function() {
    var input = $('#search-input');
    var suggestUrl = input.data('suggest');

    input.typeahead({
        minLength: 2, // will start autocomplete after 2 characters
        items: 5, // will display 5 suggestions
        highlighter: function (item) {
            var elem = this.reversed[item];
            var html = '<div class="typeahead">';

            if (elem.name) {
                html += '<div class="suggestion">' + elem.name + '</div>';
            }

            html += '</div>';

            return html;
        },
        source: function(query, process) {
            // "query" is the search string
            // "process" is a closure which must receive the suggestions list

            var $this = this;
            $.ajax({
                url: suggestUrl,
                type: 'GET',
                data: {
                    q: query
                },
                success: function(data) {
                    //  "name" is the string to display in the suggestions

                    // this "reversed" array keep a temporarly relation between each suggestion and its data
                    var reversed = {};

                    // here we simply generate the suggestions list
                    var suggests = [];

                    $.each(data, function(id, elem) {
                        reversed[elem.name] = elem;
                        suggests.push(elem.name);
                    });

                    $this.reversed = reversed;

                    // displaying the suggestions
                    process(suggests);
                }
            })
        },
        // this method is called when a suggestion is selected
        updater: function(item) {
            // do whatever you want

            return elem.name; // this return statement fills the input with the selected string
        },
        // this method determines which of the suggestions are valid. We are already doing all this server side, so here just
        // return "true"
        matcher: function() {
            return true;
        }
    });
});

 

9. Yay ! You did it !

You have now a wonderful autocomplete search input ! Congrats !

You can try to type Ir to see Iron Man in automatically suggested.

 

I hope you enjoyed this article.

Thank you for reading.

Hugo.