We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose.You can read our privacy policies and terms of use etc by clicking here.
Elasticsearch by OpenDistro
Installation for OpenDistro elasticsearch.
OpenDistro Elasticsearch Installation
Initial Configuration
Copy default fluentd and opendistro helm config files (and edit if desired)
cp ./seldon-deploy-install/prerequisites-setup/efk/fluentd-values.yaml fluentd-values.yaml
cp ./seldon-deploy-install/prerequisites-setup/efk/values-opendistro.yaml values-opendistro.yaml
Ensure Required Namespaces Exist
We’ll be installing in the seldon-logs
namespace. We’ll also set up some config in the seldon-system
namespace.
kubectl create namespace seldon-logs || echo "namespace seldon-logs exists"
kubectl create namespace seldon-system || echo "namespace seldon-system exists"
Create Secrets
Auth is required for OpenDistro. That means components interacting with it will need secrets, including seldon deploy.
We’ll need secrets in the seldon-logs
namespace (for a request logger) and seldon-system
for deploy.
Using the defaults we can set:
ELASTIC_USER=admin
ELASTIC_PASSWORD=admin
kubectl create secret generic elastic-credentials -n seldon-logs \
--from-literal=username="${ELASTIC_USER}" \
--from-literal=password="${ELASTIC_PASSWORD}" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic elastic-credentials -n seldon-system \
--from-literal=username="${ELASTIC_USER}" \
--from-literal=password="${ELASTIC_PASSWORD}" \
--dry-run=client -o yaml | kubectl apply -f -
Using alternative creds is covered in the docs for opendistro and can be set on kibana and in the fluentd values file.
Elasticsearch
Install Elasticsearch and Kibana using the following script
git clone https://github.com/opendistro-for-elasticsearch/opendistro-build
cd opendistro-build/helm/opendistro-es/
git fetch --all --tags
git checkout tags/v1.13.2
helm package .
helm upgrade --install elasticsearch opendistro-es-1.13.2.tgz --namespace=seldon-logs --values=../../../values-opendistro.yaml
You can wait for it to come up with
kubectl rollout status -n seldon-logs deployment/elasticsearch-opendistro-es-kibana
Fluentd
Install fluentd with this:
helm upgrade --install fluentd fluentd-elasticsearch --version 9.6.2 --namespace=seldon-logs --values=../../../fluentd-values.yaml --repo https://kiwigrid.github.io
Note that if alternative creds are used then these need to be set in the fluentd helm values file.
Configure Seldon Deploy
Following helm
values needs to be set in deploy-values.yaml
when using Elastisearch by OpenDistro:
requestLogger:
create: true
elasticsearch:
host: elasticsearch-opendistro-es-client-service.seldon-logs.svc.cluster.local
port: "9200"
protocol: https
elasticsearch:
basicAuth: true
url: https://elasticsearch-opendistro-es-client-service.seldon-logs.svc.cluster.local:9200
Configure EFK Ingress (Optional)
Kibana
It can be useful to access kibana’s UI without having to port-forward.
To do this create the following VirtualService
for Kibana
to enable its ingress
cat << EOF > kibana-vs.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kibana
namespace: seldon-logs
spec:
gateways:
- istio-system/seldon-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /kibana/
rewrite:
uri: /
route:
- destination:
host: elasticsearch-opendistro-es-kibana-svc
port:
number: 443
EOF
Kubectl apply -f kibana-vs.yaml
Verify Install
You can go to /kibana/ from the istio ingress endpoint (the external ip/host of istio-ingressgateway in istio-system namespace).
Or you can verify elastic direct. First port-forward:
kubectl port-forward -n seldon-logs svc/elasticsearch-opendistro-es-client-service 9200
Then verify with below, using your user and pass (a browser can also be used).
curl -XGET -k https://localhost:9200 -u admin:admin