KFserving Installation

Install and configure KFServing

This section will walk you through installation procedure of KFserving such that it is ready to be used with Seldon Deploy.

Note

Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.

This page also assumes that main Seldon components and Istio are installed.

Install KNative Serving

Knative Serving is required for KFServing. See Knative section for steps.

Install KFServing

Clone KFserving repository and checkout the right branch

KFSERVING_VERSION=v0.4.1
KFSERVING_RESOURCE_DIR=kfserving-${KFSERVING_VERSION}

git -C ${KFSERVING_RESOURCE_DIR} fetch origin || git clone https://github.com/kubeflow/kfserving.git ${KFSERVING_RESOURCE_DIR}
#have to checkout commit rather than tag as v0.4.1 yaml is missing in v0.4.1 tag
git -C ${KFSERVING_RESOURCE_DIR} checkout f253dd529ba25c080ca458d8dc36fd0a19bbd473

Install and configure KFserving

cd ${KFSERVING_RESOURCE_DIR}
kubectl apply --validate=false -f ./install/${KFSERVING_VERSION}/kfserving.yaml || true
sleep 3
kubectl patch cm -n kfserving-system inferenceservice-config -p '{"data":{"explainers":"{\n    \"alibi\":{\n        \"image\" : \"docker.io/seldonio/kfserving-alibi\",\n        \"defaultImageVersion\" : \"0.4.1\"\n    }\n}","ingress":"{\n    \"ingressGateway\" : \"istio-system/seldon-gateway\",\n    \"ingressService\" : \"istio-ingressgateway.istio-system.svc.cluster.local\"\n}","logger":"{\n    \"image\" : \"gcr.io/kfserving/logger:v0.4.0\",\n    \"memoryRequest\": \"100Mi\",\n    \"memoryLimit\": \"1Gi\",\n    \"cpuRequest\": \"100m\",\n    \"cpuLimit\": \"1\",\n    \"defaultUrl\": \"http://broker-ingress.knative-eventing.svc.cluster.local/seldon-logs/default\"\n}"}}'
# Create self-signed certs because we dont have cert-manager
./hack/self-signed-ca.sh

Configure Seldon Deploy

Enable KFserving in Seldon Deploy by adding a following section to your deploy-values.yaml files

kfserving:
  protocol: "http"
  enabled: true

Note

Note that if your istio Gateway uses https then set protocol to https.

Last modified April 20, 2021