Getting a Certificate required for Secrets Provider in Kubernetes

Secrets Management - Conjur, Secrets Hub & CP Guides and HowTo's
1

Good Day,

I am trying to set up an Init Container Kubernetes Secrets Provider in my docker Kubernetes environment. I am trying to run the command: openssl s_client --showcerts --connect https://conjur.demo.com:9443 < /dev/null > conjur.pem
I have added the conjur.demo.com to my etc/hosts file with the Server IP but it does not seem to connect to retrieve the certificate. Is there anything that I am missing?

Any advice will be greatly appreciated.

2

Hi, @quinzylam !

Could you provide the results of this failing command, or any details on your Conjur configuration?

My first thought:
openssl s_client -connect expects a hostname which doesn’t define the scheme.
For example:

openssl s_client -showcerts -connect https://example.org:443 < /dev/null returns the following:

getaddrinfo: nodename nor servname provided, or not known
connect:errno=22

While openssl s_client -showcerts -connect example.org:443 < /dev/null connects successfully:

CONNECTED(00000005)
write to ...
3

Hi John, Thank you for your response.

I have essentially tried both. The responses are as follows

Without the https

2021-08-22
2021-08-221115×149 18 KB

and with the https

2021-08-22 (1)
2021-08-22 (1)1115×111 11.7 KB

My environment is basically a replication of the getting started with secure Kubernetes Secure Kubernetes Secrets | Conjur

I am running this on a Windows docker instance with Kubernetes enables and I am managing Kubernetes and Conjur with Ubuntu for Windows. Till thus far I have had no issues with connecting to Conjur or the local Kubernetes instance. The only issue I am having it seems is to connect to the pod’s endpoints as per the getting started tutorial so I am thinking it might be a related problem but I am not 100% sure.

4

Hmm, interesting.
errno=110 seems to indicate that port 9443 is not open to connection.

Could you try curl -k -v https://conjur.demo.com:9443, and see if it’s able to connect?

5

I believe you are right in saying that it is not open, how would I go about opening it?

image
image808×206 14.8 KB

6

Alright, I think I’ve got a solution for your problem here.

Problem

I believe what you’re experiencing here is due to the helm install command in Step 7 of the interactive Secretless Broker tutorial, where it specifies that the conjur-oss service should not be externally accessible:

helm install conjur-cluster cyberark/conjur-oss \
     --set ssl.hostname=conjur.demo.com,dataKey="$(docker run --rm cyberark/conjur data-key generate)",authenticators="authn-k8s/dev" \
     --set postgres.persistentVolume.create=false \
-->  --set service.external.enabled=false \
     --namespace conjur-server

I deployed a Conjur Open Source server to my own Kubernetes-enabled Docker cluster with the command from the tutorial, which created the following services:

NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
conjur-cluster-conjur-oss   NodePort    10.97.143.80   <none>        443:30248/TCP   7m6s
conjur-cluster-postgres     ClusterIP   10.99.57.104   <none>        5432/TCP        7m6s

Note that neither has an EXTERNAL-IP.

The URL gathered in Step 7 of the tutorial, and aliased to conjur.demo.com is an internal address, accessible from inside the K8s cluster, but not from outside. When trying to reach the service with curl or openssl, with the URL gathered from Step 7:

% curl -k -v https://10.1.0.7:9443
*   Trying 10.1.0.7...
* TCP_NODELAY set
* Connection failed
* connect to 10.1.0.7 port 9443 failed: Operation timed out
* Failed to connect to 10.1.0.7 port 9443: Operation timed out
* Closing connection 0
curl: (7) Failed to connect to 10.1.0.7 port 9443: Operation timed out

% openssl s_client -showcerts -connect 10.1.0.7:9443 < /dev/null
connect: Operation timed out
connect:errno=60

Solution

I used helm upgrade to enable an external service:

helm upgrade conjur-cluster cyberark/conjur-oss \
     --set ssl.hostname=conjur.demo.com,dataKey="$(docker run --rm cyberark/conjur data-key generate)",authenticators="authn-k8s/dev" \
     --set postgres.persistentVolume.create=false \
-->  --set service.external.enabled=true \
     --namespace conjur-server

This will create an externally-accessible load-balancer service:

NAME                                TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
conjur-cluster-conjur-oss           NodePort       10.97.143.80     <none>        443:30248/TCP   11m
conjur-cluster-conjur-oss-ingress   LoadBalancer   10.105.110.217   localhost     443:31606/TCP   26s
conjur-cluster-postgres             ClusterIP      10.99.57.104     <none>        5432/TCP        11m

If you adjust the /etc/host entry to map conjur.demo.com to localhost, accessing with curl and openssl:

% curl -v -k https://conjur.demo.com:443
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to conjur.demo.com (127.0.0.1) port 443 (#0)
...
< HTTP/1.1 200 OK

% openssl s_client -showcerts -connect conjur.demo.com:443 < /dev/null
CONNECTED(00000003)
depth=1 CN = conjur-oss-ca
...

Let me know if this helps, or doesn’t!

2 Likes
7

Thank you very much John,

That was of great help.

I got around it by doing port forwarding but that was only a temporary fix this is a permanent fix and works on my cloud Kubernetes cluster as well :smile:

I truly appreciate your help and detailed explanation.

Kind Regards,
Quinten Lamprecht

3 Likes