How to reindex Magento from the command line using Terminal

The short version.

  • There is a very quick and simple way to reindex all or some of your Magento indexes using the command line
  • This is a particualrly useful trick if you cannot refresh your indexes using the Admin Panel
  • For example, a change in your Magento base urls will require a reindex but you may not have access to the Admin Panel until the catalog_url index has been updated

The detail.

In order to reindex using the command line, you need to have SSH access to your server.

If you run your own hosting on a dedicated server, then this won’t be a problem. However, if you are on shared hosting, you are most likely not to have SSH access.

Macs have a program called “Terminal” built in. Simply hit the cmd button on your keyboard followed by the spacebar and type “Terminal”.

Terminal

Terminal

A Terminal window will open and you will see your username followed by a $.

To log into your server, type the following:

ssh [user_name]@[your_server_IP_address]

Most of the time you will not want to login as “root” but I will use that as the username for this explanation.

-p 22 ssh root@123.123.123.123

Hit “Enter”

You will see the following:

[user_name]@[your_server_IP_address]’s password

For example: root@123.123.123.123’s password:

Type in the server password associated with the user.

Top tip.

When you are typing your password, you will not see anything. Don’t worry – your password will be being entered.

When you have successfully entered your password, you will see:

[user_name]@server[server_ip_address] [~]#
For examle: root@server123-123-123-123 [~]#

You are now logged into your server.

Magento has a folder called “Shell” which contains a script called indexer.php which you need to access to set-off the reindexing. You need to navigate to this folder on the server.

For this, you need to know the full path to it. Type “cd” (change directory) followed by the path to the “Shell” folder. It will look something like this:

cd  /home/[name_of_site_folder]/public_html/shell/

Hit Enter and you will see:

[user_name]@server[server_ip_address] [/home/[name_of_site_folder]/public_html/shell/#

For example: root@server123-123-123-123 [/home/example/public_html/shell]#

Top tips.

If you run cpanel (or similar) you can use it to find the path to the “Shell” folder. Simply log into cpanel and you will see /home/[name_of_site_folder]/ at the top of the folder tree.

If you want to move back up a directory just type “cd”

To check the contents of the “Shell” folder, type:

ls -1

This asks the Terminal to show you the list of all the files and folders of the directory you are in.  It will produce this:

Shell files

Shell files

We need to work with the “indexer.php” file. Firstly, it’s a good idea to check the status of your Magento indexes. Type:

 php indexer.php –status

Then press “Enter”. This produces a list of the Magento indexes and their current state.

Magento indexes

Magento indexes

Now you can choose to run a single, index, a group of indexes or all of them.

Running a single index.

Type:

php indexer.php –info

Then press “Enter” to give you a list of the names for each index that you need to use to set of the reindex procedure.

Index names

Index names

Now type:

php indexer.php –reindex [name_of_the_index]
For example: php indexer.php –reindex catalog_url

You need to use the name from the list shown above; the name with the underscores. Hitting “Enter” will start the reindex. During this time, you will simply see the cursor in the Terminal.

Reindexing of a large index such as the catalog_url index can take several minutes so just leave Terminal open until you see the success message which will look like this:

“Catalog URL index rebuilt successfully”

Running multiple indexes.

To run multiple indexing, you follow the steps steps as you would for running a single index and simply add more index names separating them with a comma. Here are the full steps:

Type:

php indexer.php –info

Then press “Enter” to give you a list of the names for each index that you need to use to set of the reindex procedure.

Index names

Index names

Now type:

php indexer.php –reindex [name_of_the_index],[name_of_the_index],
For example: php indexer.php –reindex catalog_url,cataloginventory_stock

You need to use the names from the list shown above; the names with the underscores. Hitting “Enter” will start the reindex. During this time, you will simply see the cursor in the Terminal.

Reindexing of a large index such as the catalog_url index can take several minutes so just leave Terminal open until you see the success message which will look like this:

“Catalog URL index rebuilt successfully”

An individual sucess message will be shown for each index that you asked to be reindexed.

Running all indexes.

If you want to reindex all of Magento’s indexing then use the following command:

php indexer.php –reindexall

You will need to wait for a success message for every index until the process is complete.

Logging out of your server safely.

Once you have run the indexes successfully, you need to carefully log out of your server. To do this type exit  into Terminal.

You will then see a message:

Connection to [your_server_ip_address] closed.

Magento performance – Catalog URL rewrites

The short version.

    • If your Catalog URL index needs refreshing more frequently than normal, you possibly have an issue
    • Look out for you page speed suffering (we use Google Analytics and PageSpeed to keep an eye on this)
    • Truncating (clearing out) the Catalog URL index tables (core_url_rewrite table) in the database is the only way to fix it and restore site performance
    • It’s not a Magento bug and there is no “fix”. You just need to ensure that you keep an eye on the number of rows in Catalog URL table and clear it our regularly.
    • Once you have cleared out core_url_rewrite table, you need to run the Catalog URL index via the Admin Panel

 

The detail.

There are a ton of resources and posts about Magento performance as the platform is well known for having to be optimised in order to run really well. There will be many blog posts to come on the topic explaining what I have found works well and what didn’t…

However, I wanted to share my recent experience with a really tricky performance bug that we encountered.

We made some amends to the homepage design and as soon as they went live, the site started to grind to a halt. Our server response and page load times were simply awful and we had no idea why.

We went off on a chase trying to work out why the homepage redesign had “broken” the site speed, investigating both the changes we had made and the front-end page caching.

It turns out that the site performance degradation following the homepage redesign was nothing but a distraction. The issue was actually with the Catalog URL rewrites.

The clue was in the Admin Panel. For the weeks prior to the site grinding to a halt, we’d noticed that the Catalog URL reindex needed to be refreshed quite frequently; at least a few times an hour. The index was taking up to 5 minutes to refresh and that did not seem right.

We have thousands of products and many of those are in multiple categories. The Catalog URL rewrites index in Magento match descriptive URLs to the resources so you can imagine it has a lot of work to do! Here’s a great blog post about Magento’s URL rewriting.

We cleared out the core_url_rewrite table, reindexed the Catalog URL rewrites using the Admin Panel and the site speed issue simply disappeared.

The best way we have come up with to manage this is to simply check the core_url_rewrite table regularly and watch out for the indexing messaging popping up more than usual.