Sunday, October 22, 2017

Elasticsearch Setup


To setup secure Elasticsearch local setup with Kibana on Windows with powershell.

0. Create Yaml file use with the certutil --in flag named as instances.yml

instances:
  - name: Apollo
    dns: ['localhost']
  - name: devws-kibana
    dns: ['localhost']

1.    Declare variables for use in powershell command

$root = "D:\Tools\E\elasticsearch"
[Version]$esVersion = "6.2.2"
$es = "$root\elasticsearch-$($esVersion.ToString())"
$esService = "elasticsearch_$($esVersion.ToString() -replace '\.','')"
[Version]$KibanaVersion = "6.2.2"
$kibana = "$root\kibana-$($KibanaVersion.ToString())-windows-x86_64"
$kibanaService = "elasticsearch-kibana$($KibanaVersion.ToString() -replace '\.','')"

2.    Install x-pack in elasticsearch


&"$es\bin\elasticsearch-plugin.bat" install x-pack --batch

3.    Create a Self Signed CA certificate.


&"$es\bin\x-pack\certutil.bat" ca --silent --pass password --ca-dn "CN=Elasticsearch-DevWS" --pem --out "$root\elastic-stack-ca.zip"
Expand-Archive -Path "$root\elastic-stack-ca.zip" -DestinationPath "$root\certs"

4.    Create a cert for elasticsearch and kibana


&"$es\bin\x-pack\certutil.bat" cert --silent --pem --ca-cert "$root\certs\ca\ca.crt" --ca-key "$root\certs\ca\ca.key" -in "$root\instances.yml" --ca-pass password --pass password --out "$root\certificate-bundle.zip"
Expand-Archive -Path "$root\certificate-bundle.zip" -DestinationPath "$root\certs"

5.    Copy Certs to proper directories


mkdir "$es\config\certs"
Copy-Item -Path "$root\certs\ca\ca.crt" -Destination "$es\config\certs\ca.crt"
Copy-Item -Path "$root\certs\Apollo\*" -Destination "$es\config\certs\"
mkdir "$kibana\config\certs"
Copy-Item -Path "$root\certs\ca\ca.crt" -Destination "$kibana\config\certs\ca.crt"
Copy-Item -Path "$root\certs\devws-kibana\*" -Destination "$kibana\config\certs\"

6.    Update Elasticsearch.yml to below

cluster.name: DiiConsentes
node.name: Apollo
network.host: localhost
http.port: 9210
discovery.zen.ping.unicast.hosts: [ 'localhost' ]
processors: 2
node.master: true
node.data: true
node.max_local_storage_nodes: 1
xpack.ssl.key: certs/Apollo.key
xpack.ssl.certificate: certs/Apollo.crt
xpack.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.http.ssl.enabled: true
xpack.ssl.key_passphrase: password

7.    Add secure key passphrase to keystore

"password" |  &"$es\bin\elasticsearch-keystore.bat" add xpack.ssl.secure_key_passphrase --stdin
&"$es\bin\elasticsearch-keystore.bat" list

8.    Start Elasticsearch

9.    Set password for build in accounts

$url = "https://localhost:9210/"
$output = & cmd.exe /C "$es\bin\x-pack\setup-passwords.bat auto --url $url -batch" 2>&1
$ Write-Host -ForegroundColor Green -BackgroundColor Black  $output
Output:
12:11:58.636 [main] WARN  org.elasticsearch.deprecation.common.settings.Settings - [key_passphrase] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version. Changed password for user kibana PASSWORD kibana = U6QvLg7w4Cy3pNCCMCMm  Changed password for user logstash_system PASSWORD logstash_system = LPBjX3huDyhPKFFAjuP2 Changed password for user elastic PASSWORD elastic = K7gET7kaBDWw73tm8PEA

 

10.    Parse passwords from response and save to temp files.

$elasticPassword = ($output | Select-String -Pattern "^PASSWORD\selastic\s=\s(.*)$" -AllMatches).Matches[0].Groups[1].Value
$kibanaPassword = ($output | Select-String -Pattern "^PASSWORD\skibana\s=\s(.*)$" -AllMatches).Matches[0].Groups[1].Value
$elasticPassword | Out-File -FilePath "$es\config\elastic.password" -Encoding utf8
$kibanaPassword | Out-File -FilePath "$kibana\config\kibana.password" -Encoding utf8

11.    Remove setting xpack.ssl.key_passphrase from Elasticsearch.yml

12.    Restart Elasticsearch

13.    Verify Elasticsearch is work (and it is)

14.    Install X-Pack in kibana

&"$kibana\bin\kibana-plugin.bat" install x-pack

15.    Update Kibana.yml to below

server.name: devws-kibana
server.host: localhost
elasticsearch.url: https://localhost:9210/
elasticsearch.username: kibana
elasticsearch.password:
K7gET7kaBDWw73tm8PEA
elasticsearch.ssl.certificateAuthorities: ../config/certs/ca.crt

16.    Start Kibana

17.    Verify Kibana is running and I am able to log in with elastic user

18.    Stop Kibana

19.    Update Kibana.yml to below [update all path to absolute one]

server.name: devws-kibana
server.host: localhost
server.ssl.enabled: true
server.ssl.certificate: ../config/certs/devws-kibana.crt
server.ssl.key: ../config/certs/devws-kibana.key
elasticsearch.url: https://localhost:9210/
elasticsearch.username: kibana
elasticsearch.password: K7gET7kaBDWw73tm8PEA
elasticsearch.ssl.certificateAuthorities: ../config/certs/ca.crt
xpack.security.encryptionKey: 3qrb1xee9ue9rrh3p93ykj28otgp676iu0l8ziifjopfov6h4sv9jhyp49gpm90t

20.    Generate jks file

keytool -import -v -trustcacerts -alias server-alias -file "$root\certs\ca.crt" -keystore "$root\certs\cacerts.jks" -keypass changeit -storepass changeit

Friday, April 7, 2017

Questions & Answers

Java Q&A

  1. equals() implemented and hashCode() not, Is there any issue, if no collection in use?
  2. Difference b/w JAX-WS ans JAX-RS
  3. Difference b/w  load() and get()
  4. How to test database by unittest.
  5. Which RunTimeException may be thrown by java.lang.wait() method (java.lang.IllegalMonitorStateException)
  6.  Exceptions in finalize method are ignored.
  7. what is Dynamic binding?
    Dynamic Binding refers to the case where compiler is not able to resolve the call and the binding is done at run-time only. (as overridden method call decided by run-time object)
  8. If we have 3 interfaces A,B,C which are having common method - display() and we are implementing these 3 interfaces in interface D then how this will be handled and which display() will be implemented.
    • The first interface in order must be implemented
  9. How to make a class non extendable?
    • Make the class as Final
  10. What is difference between ClassNotFoundException and ClassDefNotFoundException?
    • ClassNotFoundException: class not found at run-time in class-path using class.forName(...) or loadClass() methods.
    • ClassDefNotFoundException: class not found at run-time in class-path and in present at compile time. class is referenced with Java’s “new” operator (i.e. static loading).
  11. What is difference between HashMap, HashTable & ConcurrentHashMap?
    • HashTable: maps keys to values. Any non-null object can be used as a key or as a value.
    • HashMap: The HashMap class is roughly equivalent to Hashtable, except that it is unsynchronized (Not Thread-safe) and permits nulls
    • ConcurrentHashMap: Even though all operations are thread-safe, retrieval operations do not entail locking, and there is not any support for locking the entire table in a way that prevents all access. This class is fully interoperable with Hashtable in programs that rely on its thread safety but not on its synchronization details.
      Conclusion: If a thread-safe implementation is not needed, it is recommended to use HashMap in place of Hashtable. If a thread-safe highly-concurrent implementation is desired, then it is recommended to use ConcurrentHashMap in place of Hashtable.
  12. What is Error ?
    1. Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch.
  13. How hashCode is calculated for custom object like Employee.