NoMethodError error thrown on Conjur OSS kubernetes authenticator request

I installed conjur oss on kubernetes cluster using helm v3 & have loaded below policies.

---
# Initializes the users of the myapp application
- !group admin
- !group devops
- !group secrets_admin
- !group developer

- !grant
  role: !group admin
  members:
  - !group devops

- !grant
  role: !group secrets_admin
  members:
  - !group devops
  - !group developer

- !user Ted

- !grant
  role: !group devops
  members:
  - !user Ted

- !user John

- !grant
  role: !group developer
  members:
  - !user John
---
# This policy defines a layer of whitelisted identities permitted to authenticate to the authn-k8s endpoint.
- !policy
  id: conjur/authn-k8s/myapp/apps
  owner: !group devops
  annotations:
    description: Identities permitted to authenticate
  body:
  - !layer
    annotations:
      description: Layer of authenticator identities permitted to call authn svc
  - &hosts
    - !host
      id: myapp/*/*
      annotations:
        kubernetes/authentication-container-name: authenticator
        openshift: "false"
    - !host
      id: myapp-cloud/deployment/api
      annotations:
        kubernetes/authentication-container-name: authenticator
        kubernetes: "true"
    - !host
      id: myapp-cloud/deployment/ui
      annotations:
        kubernetes/authentication-container-name: authenticator
        kubernetes: "true"

  - !grant
    role: !layer
    members: *hosts
---
# This policy defines an authn-k8s endpoint, CA creds and a layer for whitelisted identities permitted to authenticate to it
- !policy
  id: conjur/authn-k8s/myapp
  owner: !group admin
  annotations:
    description: authentication webservice definition myapp
  body:
  - !webservice
    annotations:
      description: authn service for cluster

  - !policy
    id: ca
    body:
    - !variable
      id: cert
      annotations:
        description: CA cert for Kubernetes Pods.
    - !variable
      id: key
      annotations:
        description: CA key for Kubernetes Pods.

  # define layer of whitelisted authn ids permitted to call authn service
  - !layer users

  - !permit
    resource: !webservice
    privilege: [ read, authenticate ]
    role: !layer users

- !grant
  role: !layer conjur/authn-k8s/myapp/users
  members:
    - !layer conjur/authn-k8s/myapp/apps
---
- !policy
  id: myapp
  owner: !group devops
  annotations:
    description: This policy connects authn identities to an application identity. It defines a layer named for an application that contains the whitelisted identities that can authenticate to the authn-k8s endpoint. Any permissions granted to the application layer will be inherited by the whitelisted authn identities, thereby granting access to the authenticated identity.
  body:
  - !layer

 # add authn identities to application layer so authn roles inherit app's permissions
  - !grant
    role: !layer
    members:
    - !layer /conjur/authn-k8s/myapp/apps
---
# This policy defines the variables for myapp app
- !policy
  id: myapp-app-vars
  owner: !group secrets_admin
  annotations:
    description: This policy contains the variables for myapp

  body:
    - &variables
      - !variable aws/access_key_id
      - !variable aws/secret_access_key

    - !permit
      role: !layer /myapp
      privileges: [ read, execute ]
      resources: *variables
- !policy
  id: some-apps
  owner: !group devops
  annotations:
    description: Identities permitted to authenticate
  body:
  - !layer
    annotations:
      description: Layer of authenticator identities permitted to call authn svc
  - &hosts
    - !host
      id: myapp-cloud/*/*
      annotations:
        kubernetes/authentication-container-name: authenticator
        openshift: "false"

  - !grant
    role: !layer
    members: *hosts

# Inherit test-app's permissions
- !grant
  role: !layer myapp
  members: !layer some-apps

# Allow the host to authenticate with the authn-k8s authenticator
- !grant
  role: !layer conjur/authn-k8s/myapp/users
  members: !layer some-apps

When the kubernetes authenticator deployed as init container tries to authenticate it throws below error:

INFO: 2020/08/14 12:36:57 main.go:45: CAKC006I Authenticating as user '&{host/conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api host.conjur.authn-k8s.myapp.apps myapp-cloud.deployment.api}'
INFO: 2020/08/14 12:36:57 authenticator.go:181: CAKC005I Trying to login Conjur...
INFO: 2020/08/14 12:36:57 authenticator.go:113: CAKC007I Logging in as user &{host/conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api host.conjur.authn-k8s.myapp.apps myapp-cloud.deployment.api}.
INFO: 2020/08/14 12:36:57 requests.go:23: CAKC011I Login request to: https://rebrand-conjur.labgsd.com/authn-k8s/myapp/inject_client_cert
ERROR: 2020/08/14 12:36:57 authenticator.go:133: CAKC029E Received invalid response to certificate signing request. Reason: status code 401, 
ERROR: 2020/08/14 12:36:57 authenticator.go:184: CAKC015E Login failed
ERROR: 2020/08/14 12:36:57 main.go:48: CAKC016E Failed to authenticate

I have enabled debug level logs on conjur server, where I see below log statements:

Started POST "/authn-k8s/myapp/inject_client_cert" for 127.0.0.1 at 2020-08-14 11:37:51 +0000
Processing by AuthenticateController#k8s_inject_client_cert as HTML
  Parameters: {"service_id"=>"myapp"}
  e[1me[35mSequel::Postgres::Database (0.2ms)e[0m  BEGIN
CONJ00028D Setting common name to host.conjur.authn-k8s.myapp.apps.myapp-cloud.deployment.api
  e[1me[36mSequel::Postgres::Database (0.4ms)e[0m  e[1mSELECT * FROM "resources" WHERE "resource_id" = 'default:webservice:conjur/authn-k8s/myapp'e[0m
CONJ00027D Host id default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api extracted from CSR common name
  e[1me[35mSequel::Postgres::Database (0.4ms)e[0m  SELECT * FROM "resources" WHERE "resource_id" = 'default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api'
  e[1me[36mSequel::Postgres::Database (0.7ms)e[0m  e[1mSELECT * FROM "roles" WHERE "role_id" = 'default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api'e[0m
  e[1me[35mSequel::Postgres::Database (1.1ms)e[0m  SELECT * FROM is_role_allowed_to('default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api', 'authenticate', 'default:webservice:conjur/authn-k8s/myapp') LIMIT 1
CONJ00027D Host id default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api extracted from CSR common name
  e[1me[36mSequel::Postgres::Database (0.4ms)e[0m  e[1mSELECT * FROM "annotations" WHERE ("annotations"."resource_id" = 'default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api')e[0m
CONJ00026D Validating host id default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api
CONJ00024D Retrieved value of annotation kubernetes/authentication-container-name
CONJ00024D Retrieved value of annotation kubernetes/authentication-container-name
CONJ00015D Copying SSL certificate to myapp-cloud/api-65f76bb56b-cnnpr
CONJ00027D Host id default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api extracted from CSR common name
  e[1me[35mSequel::Postgres::Database (0.3ms)e[0m  SELECT * FROM "resources" WHERE "resource_id" = 'default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api'
  e[1me[36mSequel::Postgres::Database (0.3ms)e[0m  e[1mSELECT * FROM "annotations" WHERE ("annotations"."resource_id" = 'default:host:conjur/authn-k8s/myapp/apps/myapp-cloud/deployment/api')e[0m
  e[1me[35mSequel::Postgres::Database (0.3ms)e[0m  SELECT * FROM "resources" WHERE "resource_id" = 'default:variable:conjur/authn-k8s/myapp/ca/cert'
  e[1me[36mSequel::Postgres::Database (0.3ms)e[0m  e[1mSELECT * FROM "secrets" WHERE ("secrets"."resource_id" = 'default:variable:conjur/authn-k8s/myapp/ca/cert') ORDER BY "version" DESC LIMIT 1e[0m
Authentication Error: #<NoMethodError: undefined method `value' for nil:NilClass>

Please help.

1 Like

Hey @sameer - can you share which version of Conjur OSS you are running? I would like to try to reproduce this error to determine why you’re seeing it.

If you need help determining the version, please let me know. Recent releases will print the version on startup (so that it would show in the Docker logs) or show it on the homepage if you visit the URL of your Conjur instance in a browser.

Hi @izgerij I’m running conjur OSS v5.0 & with kubernetes authenticator v1.5

@sameer to be clear, how did you find these versions? What version / flavor of Kubernetes are you running on?

Hey @sameer! Can you print the environment variables of the OSS pod/container?

Thanks,

Darren

Hi @sameer, just checking to make sure that you were able to get the issue you were experiencing resolved. Chris

Hi All, Thanks for your replies. I found the issue, which was that I had not loaded the mTLS certs in Conjur.

I think the error log, at least on the server, should give a hint around those lines. I don’t understand ruby much, but the server error looks like a native ruby error rather than a custom Conjur error.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.