We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose.You can read our privacy policies and terms of use etc by clicking here.
Elasticsearch by Elastic
Installation for elastic-provided elasticsearch.
Elasticsearch Installation
Initial Configuration
Copy default fluentd-config (and edit as required)
cp ./seldon-deploy-install/prerequisites-setup/efk/fluentd-values.yaml fluentd-values.yaml
Elasticsearch
Install Elasticsearch using the following script
kubectl create namespace seldon-logs || echo "namespace seldon-logs exists"
helm upgrade --install elasticsearch elasticsearch \
--version 7.6.0 \
--namespace seldon-logs \
--set service.type=ClusterIP \
--set antiAffinity="soft" \
--repo https://helm.elastic.co \
--set image=docker.elastic.co/elasticsearch/elasticsearch-oss
kubectl rollout status statefulset/elasticsearch-master -n seldon-logs
Fluentd and Kibana
If not using auth (not by default) then set elasticsearch.auth.enabled
to false in the fluentd values file.
Install fluentd and kibana using the following script
helm upgrade --install fluentd fluentd-elasticsearch \
--version 8.0.0 \
--namespace seldon-logs -f fluentd-values.yaml \
--repo https://kiwigrid.github.io
helm upgrade --install kibana kibana \
--version 7.6.0 \
--namespace seldon-logs \
--set service.type=ClusterIP \
--repo https://helm.elastic.co \
--set image=docker.elastic.co/kibana/kibana-oss
kubectl rollout status deployment/kibana-kibana -n seldon-logs
Configure Seldon Deploy
Following helm
values needs to be set in deploy-values.yaml
when using Elastisearch by Elastic:
requestLogger:
create: true
elasticsearch:
host: elasticsearch-master.seldon-logs.svc.cluster.local
port: "9200"
protocol: http
elasticsearch:
basicAuth: false
url: http://elasticsearch-master.seldon-logs.svc.cluster.local:9200
Auth is not required for the elastic-provided version of elasticsearch. The elasticsearch.basicAuth
option in the seldon deploy helm chart can be set to false
, as illustrated above.
Authentication
If authentication is required to access your ElasticSearch cluster, you will
need to configure your credentials so that Seldon can access it.
To do this, you can provide your ElasticSearch user and password through a
secret.
By default, Seldon will look for a secret named elastic-credentials
.
As an example, if we assume that ElasticSearch can be accessed using the
admin
/ admin
credentials, we could create the relevant secrets as:
ELASTIC_USER=admin
ELASTIC_PASSWORD=admin
kubectl create secret generic elastic-credentials -n seldon-logs \
--from-literal=username="${ELASTIC_USER}" \
--from-literal=password="${ELASTIC_PASSWORD}" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic elastic-credentials -n seldon-system \
--from-literal=username="${ELASTIC_USER}" \
--from-literal=password="${ELASTIC_PASSWORD}" \
--dry-run=client -o yaml | kubectl apply -f -
Configure EFK Ingress (Optional)
Kibana
It can be useful to access kibana’s UI without having to port-forward.
To expose kibana externally it needs to have its own path. This means a custom values file:
extraEnvs:
- name: SERVER_BASEPATH
value: "/kibana"
That should be referenced with -f
as an additional parameter on the previous helm install command.
Then create a following VirtualService
for Kibana
to enable its ingress
cat << EOF > kibana-vs.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kibana
namespace: seldon-logs
spec:
gateways:
- istio-system/seldon-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /kibana/
rewrite:
uri: /
route:
- destination:
host: kibana-kibana
port:
number: 5601
EOF
Kubectl apply -f kibana-vs.yaml