Editing OpenSearch Cluster Installation (section)

=== <code>opensearch.yml</code> ===
While the OpenSearch software can be rather forgiving for low-security environments, if you turn on full hostname verification, it is rather strict.  The biggest thing to remember is that '''SAN entries in the node certificates aren't always respected.'''  What this boils down to is that if full verification is turned on, you cannot use IP addresses in the <code>opensearch.yml</code> (or any other configuration file such as the logstash pipelines).

Each node will have a unique version of opensearch.yml due to certificate file names, the role(s) for the node, and the temperature (for data nodes).  The significant parameters in the opensearch.yml file are:
* <code>cluster.name</code> - must be identical in all nodes
* <code>node.name</code>- can be the hostname, but doesn't have to be (must be unique)
* <code>path.data</code> - data nodes only -- must be unique to the node (i.e if using shared storage, each node must have its own directory on the shared volume)
* <code>path.logs</code> - must be unique to the node (same as above)
* <code>network.host</code> -- host's FQDN where all traffic will be received
**Can be a single hostname or an array of hostnames in <code>["name", "name"]</code> format
**If you are doing transport-level host verification this CANNOT be an IP -- OpenSearch can't handle the SANs in the certificates needed to make it work
* <code>http.port</code> -- leave it as the default (<code>9200</code>) but uncomment it to lock it in
* <code>discovery.seed_hosts</code> -- FQDN for all the masters (same for all nodes) -- only used for cluster bootstrap
* <code>cluster.initial_master_nodes</code> -- put the master node names here (not the hostnames) -- same for all nodes
* <code>node.[master|data|ingest|remote]</code> - set the type of the node (specify all four)
* <code>node.attr.temp</code> - (data nodes only) defines data storage tiers (<code>hot</code>/<code>warm</code>/<code>cold</code>)
The stock configuration file has most of these specified -- edit the file to fit the configuration of the node.

The security configuration must be added next.  Since we are not using the demo security as our template, all the relevant entries must be added now.  Copy this block into the end of the stock opensearch.yml, editing as needed to provide the relevant certificate file names:
 node.master: true
 node.ingest: true
 node.data: false
 #node.remote: false
 
 #node.attr.temp: hot
 
 ######## Start OpenSearch Security Configuration ########
 plugins.security.ssl.transport.pemcert_filepath: <node>.crt
 plugins.security.ssl.transport.pemkey_filepath: <node>-pkcs8.key
 plugins.security.ssl.transport.pemtrustedcas_filepath: <CAcert>
 plugins.security.ssl.transport.enforce_hostname_verification: true
 
 plugins.security.ssl.http.enabled: true
 plugins.security.ssl.http.pemcert_filepath: <node>.crt
 plugins.security.ssl.http.pemkey_filepath: <node>-pkcs8.key
 plugins.security.ssl.http.pemtrustedcas_filepath: <CAcert>
 
 plugins.security.allow_unsafe_democertificates: false
 plugins.security.allow_default_init_securityindex: true
 
 # at least one cert -- must be an EXACT copy of the DN for the certificates to be used for admin auth
 plugins.security.authcz.admin_dn:
   - CN=Admin,O=OpenSearch,C=US
 
 # these must be EXACT copies of the DN for the node certificates
 plugins.security.nodes_dn:
   - CN=master-1.opensearch.localnet,O=OpenSearch,C=US
   - CN=data-hot-1.opensearch.localnet,O=OpenSearch,C=US
 
 # this is all boiler plate ... no modifications needed
 plugins.security.audit.type: internal_opensearch
 plugins.security.enable_snapshot_restore_privilege: true
 plugins.security.check_snapshot_restore_write_privileges: true
 plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
 plugins.security.system_indices.enabled: true
 # this next bit should ALL be on one line ...
 plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]
 
 node.max_local_storage_nodes: 3
 
 ######## End OpenSearch Security Configuration ########
Note that there are two sections where the DNs from the certificates must be provided to enable admin authentication for security configuration, and for validating the nodes that will be part of the cluster.  You can easily show the DNs for your certificates using this command:
 openssl x509 -subject -nameopt RFC2253 -noout -in <cert file name>
The DN for the admin cert and the DNs for the node certs go in separate sections as shown above.  Add all the cluster nodes into the nodes_dn section -- this must be the same for all nodes or they will not be able to communicate.