ELK – Auto-delete older Logstash indices

The following is an approach to auto-delete Logstash indices in Elasticsearch every X days. The following steps are to be run on your ELK host.

Get curator-cli

sudo pip install elasticsearch-curator -U

Create script

cd ~/
vim elasticsearch_del.sh

My preference is to delete indices older than 30 days, change the 30 to your preference. Then save the file.

#!/bin/bash                                                                                                                                           
/usr/local/bin/curator_cli "$@" delete_indices --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":30},{"filtertype":"pattern","kind":"prefix","value":"logstash"}]'

Now make the script executable:
chmod +x elasticsearch_del.sh

Then run the script to make sure it works – use the --dry-run argument to test (i.e. not actually take any action):

./elasticsearch_del.sh --dry-run

If you you’re happy with the output and want to run it for real:

./elasticsearch_del.sh

Setup a CRON schedule job

crontab -e

Add the following line – changing the schedule to your preference. This runs it every Saturday at 5pm:

0 17 * * SAT /home/db/elasticsearch_del.sh