Elasticsearch Cluster TLS Encryption
Enabling TLS encryption is a great add on protection for you clusters and also most of the compliance certifications need TLS/SSL encrypted communication. It encrypts, both the connection between in between nodes and HTTP API calls. It takes only few steps to enable TLS for your elasticsearch cluster as most of the hard tasks are already handled by the executables provided in elasticsearch zip. If you are not familiar with the concept of TLS then checkout this awesome blog by Andrew Howden in which he explains how TLS works in the simplest way. Now lets start the process of protecting our elasticsearch clusters🫡. We will go with using Self signed certificates for this example blog. In production you can use your companys Authority files to sign the certificates. We will be using elasticsearch-certutil to generate required certificates.
Certificate Authority(CA):
A Certificate Authority (CA) is a trusted entity that issues digital certificates. These certificates verify the ownership/authenticity of a digital asset. The CA's role is to confirm the identity of entities (such as websites) and to bind public keys with those identities through digital certificates. This helps establish trust in a digital communication environment.
X.509 Certificate:
An X.509 Certificate is a digital certificate that uses the X.509 standard to define the format of public key certificates. X.509 certificates are used to establish a secure, encrypted connection between a client (like a web browser) and a server (like a website).
Steps to generate certificates:
‣Generating a certificate authority to sign our X.509 certificates.
./bin/elasticsearch-certutil ca
Executing this will prompt for few questions.
CA file name - you can name it anything you wish to. I will leave it as default.
CA password - Its very important to remember the CA password as we need it for configuration purposes in multiple places. you can also leave it empty. For now i will go with "12345" as password.
//Download elasticsearh zip and extract the same
~/clusters/sample/elasticsearch % ./bin/elasticsearch-certutil ca
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.
The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.
Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority
By default the 'ca' mode produces a single PKCS#12 output file which holds:
* The CA certificate
* The CA's private key
If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key
Please enter the desired output file [elastic-stack-ca.p12]:
Enter password for elastic-stack-ca.p12 :
~/clusters/sample/elasticsearch % ls
LICENSE.txt README.asciidoc blog elastic-stack-ca.p12 lib modules upgrade
NOTICE.txt bin config jdk logs plugins
Now you can see that elastic-stack-ca.p12 is created. This certificate authority file contains the following components in it:
Private key
Public certificate which also contains a public key
To view the contents of the .p12 file you can use the below command
openssl pkcs12 -info -in elastic-stact-ca.p12
‣Next we are going to generate Node certificate for enabling encrypting communication in between nodes.
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
Executing this will prompt for few questions.
CA file password - you can name it anything you wish to. I will leave it as default.
Certificate password - We need to remember the Certificate password also. you can also leave it empty. For now i will go with "12345" as password.
Contents of Node Certificate:
Node's Private key
Node certificate ( also contains public key)
CA certificate (only certificate of CA, private key will not be included )
‣ Final step in certificate generation part is to generate a certificate for HTTPS communication. (You can skip this step if you do not want TLS for HTTP communications ).
./bin/elasticsearch-certutil http
Contents of Node Certificate:
HTTP's Private key
HTTP certificate ( also contains public key)
CA certificate (only certificate of CA, private key will not be included)
Both Node and HTTP certificates are X.509 certificates.
## Do you wish to generate a Certificate Signing Request (CSR)?
A CSR is used when you want your certificate to be created by an existing
Certificate Authority (CA) that you do not control (that is, you don't have
access to the keys for that CA).
If you are in a corporate environment with a central security team, then you
may have an existing Corporate CA that can generate your certificate for you.
Infrastructure within your organisation may already be configured to trust this
CA, so it may be easier for clients to connect to Elasticsearch if you use a
CSR and send that request to the team that controls your CA.
If you choose not to generate a CSR, this tool will generate a new certificate
for you. That certificate will be signed by a CA under your control. This is a
quick and easy way to secure your cluster with TLS, but you will need to
configure all your clients to trust that custom CA.
Generate a CSR? [y/N]N
## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
If you have an existing CA certificate and key, then you can use that CA to
sign your new http certificate. This allows you to use the same CA across
multiple Elasticsearch clusters which can make it easier to configure clients,
and may be easier for you to manage.
If you do not have an existing CA, one will be generated for you.
Use an existing CA? [y/N]y
## What is the path to your CA?
Please enter the full pathname to the Certificate Authority that you wish to
use for signing your new http certificate. This can be in PKCS#12 (.p12), JKS
(.jks) or PEM (.crt, .key, .pem) format.
CA Path: /Users/rohit/clusters/sample/elasticsearch/elastic-stack-ca.p12
Reading a PKCS12 keystore requires a password.
It is possible for the keystore's password to be blank,
in which case you can simply press <ENTER> at the prompt
Password for elastic-stack-ca.p12:
## How long should your certificates be valid?
Every certificate has an expiry date. When the expiry date is reached clients
will stop trusting your certificate and TLS connections will fail.
Best practice suggests that you should either:
(a) set this to a short duration (90 - 120 days) and have automatic processes
to generate a new certificate before the old one expires, or
(b) set it to a longer duration (3 - 5 years) and then perform a manual update
a few months before it expires.
You may enter the validity period in years (e.g. 3Y), months (e.g. 18M), or days (e.g. 90D)
For how long should your certificate be valid? [5y] 10y
## Do you wish to generate one certificate per node?
If you have multiple nodes in your cluster, then you may choose to generate a
separate certificate for each of these nodes. Each certificate will have its
own private key, and will be issued for a specific hostname or IP address.
Alternatively, you may wish to generate a single certificate that is valid
across all the hostnames or addresses in your cluster.
If all of your nodes will be accessed through a single domain
(e.g. node01.es.example.com, node02.es.example.com, etc) then you may find it
simpler to generate one certificate with a wildcard hostname (*.es.example.com)
and use that across all of your nodes.
However, if you do not have a common domain name, and you expect to add
additional nodes to your cluster in the future, then you should generate a
certificate per node so that you can more easily generate new certificates when
you provision new nodes.
Generate a certificate per node? [y/N]N
## Which hostnames will be used to connect to your nodes?
These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.
You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.
If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.
Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
You did not enter any hostnames.
Clients are likely to encounter TLS hostname verification errors if they
connect to your cluster using a DNS name.
Is this correct [Y/n]Y
## Which IP addresses will be used to connect to your nodes?
If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.
If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.
Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
You did not enter any IP addresses.
Is this correct [Y/n]Y
## Other certificate options
The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.
Key Name: elasticsearch
Subject DN: CN=elasticsearch
Key Size: 2048
Do you wish to change any of these options? [y/N]N
## What password do you want for your private key(s)?
Your private key(s) will be stored in a PKCS#12 keystore file named "http.p12".
This type of keystore is always password protected, but it is possible to use a
blank password.
If you wish to use a blank password, simply press <enter> at the prompt below.
Provide a password for the "http.p12" file: [<ENTER> for none]
## Where should we save the generated files?
A number of files will be generated including your private key(s),
public certificate(s), and sample configuration options for Elastic Stack products.
These files will be included in a single zip archive.
What filename should be used for the output zip file? [/Users/rohit/clusters/sample/elasticsearch/elasticsearch-ssl-http.zip]
Zip file written to /Users/rohit/clusters/sample/elasticsearch/elasticsearch-ssl-http.zip
After generating the necessary certificate copy them to elasticsearch/config/certs folder. That's all for certificate generation part. Now lets configure the same.
Configuring in elasticsearch.yaml:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: "certs/http.p12"
xpack.security.http.ssl.verification_mode: certificate
Note: If you have set password for your certificate files then you have to add it to elasticsearch's keystore. You can use the below commands to do the same.
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
//prompts for node certificates password
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
//prompts for node certificates password
./bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
//prompts for http certificates password
xpack.security.http.ssl.verification_mode / xpack.security.transport.ssl.verification_mode:
It helps to control the SSL certicate verification level. Based on this configuration we can restrict connection to a finite set of host or IP addresses. For enabling this restriction we need to provide the list of allowed hostnames and IP addresses while generation of certificate. There are three modes available. They are
full:
Validates that the provided certificate: has an issue date that’s within the not_before and not_after dates; chains to a trusted Certificate Authority (CA); has a hostname or IP address that matches the names within the certificate.certificate:
Validates the provided certificate and verifies that it’s signed by a trusted authority (CA), but doesn’t check the certificate hostname.none:
It basically disables SSL verification.
In this example we have generated single node and http certificate which we will be using in all nodes and also we have not provided IP address or hostname for certificate generation therefore we will go with certificate mode. Incase you had multiple nodes if you go with full mode cluster will not get created due to Host verification failure.