Elasticsearch by OpenDistro

OpenDistro Elasticsearch

Installation for OpenDistro elasticsearch.


OpenDistro Elasticsearch Installation

Initial Configuration

Copy default fluentd and opendistro helm config files (and edit if desired)

cp ./seldon-deploy-install/prerequisites-setup/efk/fluentd-values.yaml fluentd-values.yaml
cp ./seldon-deploy-install/prerequisites-setup/efk/values-opendistro.yaml values-opendistro.yaml

Ensure Required Namespaces Exist

We’ll be installing in the seldon-logs namespace. We’ll also set up some config in the seldon-system namespace.

kubectl create namespace seldon-logs || echo "namespace seldon-logs exists"
kubectl create namespace seldon-system || echo "namespace seldon-system exists"

Create Secrets

Auth is required for OpenDistro. That means components interacting with it will need secrets, including seldon deploy.

We’ll need secrets in the seldon-logs namespace (for a request logger) and seldon-system for deploy.

Using the defaults we can set:

ELASTIC_USER=admin
ELASTIC_PASSWORD=admin

kubectl create secret generic elastic-credentials -n seldon-logs \
  --from-literal=username="${ELASTIC_USER}" \
  --from-literal=password="${ELASTIC_PASSWORD}" \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic elastic-credentials -n seldon-system \
  --from-literal=username="${ELASTIC_USER}" \
  --from-literal=password="${ELASTIC_PASSWORD}" \
  --dry-run=client -o yaml | kubectl apply -f -

Using alternative creds is covered in the docs for opendistro and can be set on kibana and in the fluentd values file.

Elasticsearch

Install Elasticsearch and Kibana using the following script

git clone https://github.com/opendistro-for-elasticsearch/opendistro-build
cd opendistro-build/helm/opendistro-es/
git fetch --all --tags
git checkout tags/v1.13.2
helm package .
helm upgrade --install elasticsearch opendistro-es-1.13.2.tgz --namespace=seldon-logs --values=../../../values-opendistro.yaml

You can wait for it to come up with

kubectl rollout status -n seldon-logs deployment/elasticsearch-opendistro-es-kibana

Fluentd

Install fluentd with this:

helm upgrade --install fluentd fluentd-elasticsearch --version 9.6.2 --namespace=seldon-logs --values=../../../fluentd-values.yaml --repo https://kiwigrid.github.io

Note that if alternative creds are used then these need to be set in the fluentd helm values file.

Configure Seldon Deploy

Following helm values needs to be set in deploy-values.yaml when using Elastisearch by OpenDistro:

requestLogger:
  create: true
  elasticsearch:
    host: elasticsearch-opendistro-es-client-service.seldon-logs.svc.cluster.local
    port: "9200"
    protocol: https

elasticsearch:
  basicAuth: true
  url: https://elasticsearch-opendistro-es-client-service.seldon-logs.svc.cluster.local:9200

Configure EFK Ingress (Optional)

Kibana

It can be useful to access kibana’s UI without having to port-forward.

To do this create the following VirtualService for Kibana to enable its ingress

cat << EOF > kibana-vs.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: kibana
  namespace: seldon-logs
spec:
  gateways:
    - istio-system/seldon-gateway
  hosts:
    - '*'
  http:
    - match:
        - uri:
            prefix: /kibana/
      rewrite:
        uri: /
      route:
        - destination:
            host: elasticsearch-opendistro-es-kibana-svc
            port:
              number: 443
EOF

Kubectl apply -f kibana-vs.yaml

Verify Install

You can go to /kibana/ from the istio ingress endpoint (the external ip/host of istio-ingressgateway in istio-system namespace).

Or you can verify elastic direct. First port-forward:

kubectl port-forward -n seldon-logs svc/elasticsearch-opendistro-es-client-service 9200

Then verify with below, using your user and pass (a browser can also be used).

curl -XGET -k https://localhost:9200 -u admin:admin
Last modified April 19, 2021