OpenID Connect conformance-suite run¶
HelixIAM was run as an OpenID Provider (OP) against every OpenID Connect OP certification plan of the OpenID
Foundation conformance suite, with the suite playing the relying party. This is a self-run of the OpenID Foundation
conformance suite 5.3.3 (revision f0133e9), not OpenID Certification: certification is granted by the OpenID
Foundation itself and was not requested. HelixIAM is not OpenID Certified.
Outcome (2026-11-01). All 13 OP certification plans ran to the end, every module finished, with 0 failures in
413 module runs. Twelve plans ran against HelixIAM 02a648fb; the Session Management plan was re-run against 70d7a43c
(see Session Management). Every warning is listed below with its specification
reference and why it is acceptable, and every skipped module with the reason the suite gave.
| Finding | Result | Status |
|---|---|---|
W1 — ID token carries claims the suite did not request (realm_access, helix_grant, helix_token_type) |
warning in oidcc-server, 13 modules |
justified: OpenID Connect Core 1.0 §2 allows other claims |
| W2 — access token in the userinfo POST body not accepted | warning in oidcc-userinfo-post-body, 10 modules |
justified: RFC 6750 §2.2 makes that method optional |
W3 — email in the ID token when an access token is also issued |
warning in oidcc-scope-email and oidcc-alternate-happy-flow, 16 modules |
justified: OpenID Connect Core 1.0 §5.4 does not forbid it; the suite says so |
W4 — no acr in the ID token after acr_values |
warning in oidcc-ensure-request-with-acr-values-succeeds, 12 modules |
justified: a password sign-in reaches no eIDAS level; §3.1.2.1 makes acr_values voluntary |
W5 — discovery member unknown to the suite (verified_claims_supported) |
warning in oidcc-discovery-endpoint-verification, 2 modules |
justified: OpenID Connect for Identity Assurance 1.0 metadata |
| S1–S3 — 29 module runs skipped by the suite | skipped | not a deviation: features HelixIAM does not offer on purpose (none, pairwise) |
What was run¶
| Suite | registry.gitlab.com/openid/conformance-suite:latest, reporting version 5.3.3 in every exported log, revision f0133e9 (the revision is recorded in the commit that added the logs, not in the exports). Source: https://gitlab.com/openid/conformance-suite |
| HelixIAM | 02a648fb for the twelve plans other than Session Management, 70d7a43c ("fix(oidc): check_session iframe answers a window of the parent's origin, not only the parent") for the Session Management plan; both in the default (production) profile, behind a TLS-terminating nginx. 70d7a43c changes only static/js/check-session.js (and its test) on top of 02a648fb. The commits are not recorded in the exports themselves. The run took place on 2026-11-01 |
| Plans | the 13 oidcc-*-certification-test-plan OP plans of suite 5.3.3: Config, Basic, Form Post Basic, Implicit, Form Post Implicit, Hybrid, Form Post Hybrid, Dynamic, RP-Initiated Logout, Back-Channel Logout, Front-Channel Logout, Session Management, 3rd Party-Init |
| Clients | static clients client_secret_basic and client_secret_post (the plans' own variants); the Dynamic and 3rd Party-Init plans register their clients (private_key_jwt with a jwks object in the Dynamic plan) |
| Logs | docs/compliance/oidc-conformance/2026-11-01/ in the product repository: one folder per plan with the suite's JSON export of every module (gzipped, secrets removed), the screenshots uploaded for review modules (screenshots/), and results.json (per plan: suite plan id, variant, counts; per module: status, result, findings, the HelixIAM pages the browser stopped on). One folder per plan (final3-*; final3sm-session-management is the Session Management re-run) |
The harness is in e2e/oidf-conformance/ (see
Reproducing the run).
Results¶
Columns count test modules: one module run per variant, so a plan with three response types runs most modules three
times. Pass = the suite's PASSED. Pass (review) = every check passed and the suite waits for a person to
look at the evidence: the page HelixIAM showed for a bad request or a logout (oidcc-response-type-missing,
oidcc-ensure-registered-redirect-uri, the three redirect-URI tests of the Dynamic plan, eight RP-initiated logout tests), or
the login page for a test that must force a new sign-in (oidcc-prompt-login, oidcc-max-age-1) or show the
client's logo, policy or terms (oidcc-registration-logo-uri, -policy-uri, -tos-uri). The driver uploaded a
screenshot for each. Warning = passed with at least one warning.
Per plan¶
| Plan | Suite plan | Variant | Modules | Pass | Pass (review) | Fail | Warning | Skipped | Suite plan id |
|---|---|---|---|---|---|---|---|---|---|
| Config OP | oidcc-config-certification-test-plan |
discovery, static client | 1 | 0 | 0 | 0 | 1 (W5) | 0 | ZPdE9gMd7zSny |
| Basic OP | oidcc-basic-certification-test-plan |
code; discovery, static client; client_secret_basic, client_secret_post |
35 | 24 | 4 | 0 | 5 | 2 | dkvdOqKhky7mU |
| Form Post OP | oidcc-formpost-basic-certification-test-plan |
as Basic, response_mode=form_post |
35 | 24 | 4 | 0 | 5 | 2 | EouU1bQEIUsSZ |
| Implicit OP | oidcc-implicit-certification-test-plan |
id_token, id_token token; client_secret_basic |
54 | 36 | 7 | 0 | 7 | 4 | UP453EmNxXMl1 |
| Form Post Implicit OP | oidcc-formpost-implicit-certification-test-plan |
as Implicit, form_post |
54 | 36 | 7 | 0 | 7 | 4 | D5PRriTzXvED1 |
| Hybrid OP | oidcc-hybrid-certification-test-plan |
code id_token, code token, code id_token token; client_secret_basic, client_secret_post |
96 | 67 | 10 | 0 | 13 | 6 | Sfztfk1xo841K |
| Form Post Hybrid OP | oidcc-formpost-hybrid-certification-test-plan |
as Hybrid, form_post |
96 | 67 | 10 | 0 | 13 | 6 | iDih9dgbJWAy3 |
| Dynamic OP | oidcc-dynamic-certification-test-plan |
code; dynamic client, private_key_jwt |
23 | 10 | 6 | 0 | 2 | 5 | 5jukUHe8WBVQE |
| RP-Initiated Logout OP | oidcc-rp-initiated-logout-certification-test-plan |
code; static client |
11 | 3 | 8 | 0 | 0 | 0 | LDs9ZVOpg4vjQ |
| Back-Channel Logout OP | oidcc-backchannel-rp-initiated-logout-certification-test-plan |
code; static client |
2 | 2 | 0 | 0 | 0 | 0 | yqMtp5as66fqX |
| Front-Channel Logout OP | oidcc-frontchannel-rp-initiated-logout-certification-test-plan |
code; static client |
2 | 2 | 0 | 0 | 0 | 0 | 94WYlL8CPQNN5 |
| Session Management OP | oidcc-session-management-certification-test-plan |
code; static client |
2 | 2 | 0 | 0 | 0 | 0 | t8DOCPOQveprH |
| 3rd Party-Init OP | oidcc-3rdparty-init-login-certification-test-plan |
code; dynamic client |
2 | 2 | 0 | 0 | 0 | 0 | l8Hu5Q7FnL9co |
| Total (13 plans) | 413 | 275 | 56 | 0 | 53 | 29 |
Warnings by finding: W1 13, W2 10, W3 16, W4 12, W5 2 (each warning module carries exactly one of them). Skipped by reason: S1 26, S2 1, S3 2.
Earlier stalls: the login rate limit¶
A previous full run had four modules the driver stopped as stalled. The cause: the per-IP login rate limit
(RateLimitFilter) answered a late sign-in POST in the long plans with 429. It is not a product defect; the harness
now waits and retries (bounded, logged), and in this run every module of 12 plans finished on the first attempt.
Session Management fix and re-run¶
In the first full run of this build (02a648fb) the Session Management plan's oidcc-session-management-rp-initiated-logout
module did not finish (the driver stopped it after 90 seconds idle, and a repeat of just that module did the same). The
other 12 plans finished. Cause: the check_session iframe (/js/check-session.js) had been restricted to messages from
window.parent. The suite's session page embeds the OP iframe and its own RP iframe as siblings, so the RP iframe's
message came from a frame of the parent's origin that is not the parent; it got no answer and the test waited for it.
OpenID Connect Session Management 1.0 §4.2 requires the caller to have the parent frame's origin, which a sibling frame
of that origin has. It was a product defect, not a harness or suite problem. 70d7a43c answers any window of the
parent's origin (at that origin) and still ignores every other origin; a browser test covers the parent, a same-origin
sibling and a foreign frame. Only that file changed, so only the Session Management plan was re-run on 70d7a43c
(2 of 2 modules passed); the other 12 plans keep their 02a648fb results. An earlier full run on 68f68deb had passed
this plan before the iframe was restricted.
Warnings¶
Every warning the suite reported, why it appears, and why it is acceptable.
W1 — EnsureIdTokenDoesNotContainNonRequestedClaims (oidcc-server) — justified¶
"id_token contains non-requested claim 'realm_access'", the same for helix_grant and helix_token_type.
Why. HelixIAM's ID tokens carry:
- realm_access: the user's realm roles, which relying parties use for authorization;
- helix_grant: the id of the grant the token belongs to;
- helix_token_type: the type the token was issued as (id_token), which HelixIAM checks before anything else so a
token is never accepted as another type.
Why acceptable. OpenID Connect Core 1.0 §2: "ID Tokens MAY contain other Claims. Any Claims used that are not understood MUST be ignored." None of these is a claim about the user beyond the roles the realm grants. The standard claims follow the granted scopes (§5.4), and the suite's checks of those passed.
W2 — UserInfoEndpointWithAccessTokenInBodyNotSupported (oidcc-userinfo-post-body) — justified¶
"The server returned a non-2xx HTTP response, and hence does not appear to support access tokens passed in the POST
body." HelixIAM answered 401.
Why acceptable. RFC 6750 §2 lists three ways to send a bearer token. Resource servers MUST support the
Authorization header (§2.1); the form-encoded body (§2.2) is optional. OpenID Connect Core 1.0 §5.3.1 recommends
the header. The same plans' oidcc-userinfo-get and oidcc-userinfo-post-header passed. The suite reports this as a
warning, not a failure.
W3 — EnsureIdTokenDoesNotContainEmailForScopeEmail (oidcc-scope-email, oidcc-alternate-happy-flow) — justified¶
"Unexpectedly found email in id_token ... Technically returning unrequested claims does not violate the
specifications", OIDCC-5.4. It appears only for response types that also issue an access token; with
response_type=id_token the same modules pass.
Why. HelixIAM puts the claims of the granted scopes in the ID token as well as in userinfo, so a relying party that
reads only the ID token gets the email address it asked for with scope=email.
Why acceptable. OpenID Connect Core 1.0 §5.4 says the claims of these scopes are returned from userinfo when an access token is issued; it does not forbid them in the ID token, as the suite's message states. The claims are only those of a granted scope.
W4 — ValidateIdTokenACRClaimAgainstAcrValuesRequest (oidcc-ensure-request-with-acr-values-succeeds) — justified¶
"An acr value was requested using acr_values, so the server 'SHOULD' return an acr claim, but it did not.",
OIDCC-3.1.2.1, OIDCC-15.1.
Why. The suite asks for every value of acr_values_supported: the eIDAS levels low, substantial, high.
HelixIAM sets acr only when the sign-in reached such a level (a brokered eID or EUDI Wallet login). The test user
signs in with a password, which reaches none, so HelixIAM does not claim one.
Why acceptable. OpenID Connect Core 1.0 §3.1.2.1: acr_values is a voluntary request, and the module passed (the
sign-in succeeded). Returning an eIDAS level for a password sign-in would state an assurance that was not reached.
W5 — CheckForUnexpectedParametersInServerMetadata (oidcc-discovery-endpoint-verification) — justified¶
"Unknown properties were found in the OAuth Authorization Server metadata", RFC8414-2, OIDCD-3; the one unknown
property is verified_claims_supported. In the Config and Dynamic plans.
Why acceptable. verified_claims_supported is OP metadata of OpenID Connect for Identity Assurance 1.0, published
with trust_frameworks_supported, evidence_supported and claims_in_verified_claims_supported for HelixIAM's
verified_claims (wallet-verified attributes). RFC 8414 §2 and OpenID Connect Discovery 1.0 §3 allow additional
metadata. The suite tests Identity Assurance in a separate plan (ekyc-test-plan-oidccore), not run here.
Skipped modules¶
The suite skips a module when the server's discovery says it does not offer what the module tests.
| Module | Runs | The suite's reason | Why HelixIAM does not offer it | |
|---|---|---|---|---|
| S1 | oidcc-ensure-request-object-with-redirect-uri, oidcc-unsigned-request-object-supported-correctly-or-rejected-as-unsupported, oidcc-request-uri-unsigned |
26 | "'none' is not listed in request_object_signing_alg_values_supported" | Request objects must be signed (RS256); an unsigned (alg=none) request object is refused. |
| S2 | oidcc-idtoken-unsigned |
1 | "'id_token_signing_alg_values_supported' doesn't support 'none'" (the suite adds: acceptable for certification) | ID tokens are always signed. |
| S3 | oidcc-registration-sector-uri, oidcc-registration-sector-bad |
2 | "Server configuration does not explicitly support pairwise subject type" | subject_types_supported is public. |
S1 in detail: oidcc-ensure-request-object-with-redirect-uri was skipped 13 times (once per variant of the Basic,
Form Post, Implicit, Hybrid and Dynamic plans and their form_post forms),
oidcc-unsigned-request-object-supported-correctly-or-rejected-as-unsupported 12 times and
oidcc-request-uri-unsigned once (Dynamic OP): 26 in all.
Test setup¶
- Network. The suite (
oidfop-suite-server,oidfop-suite-nginx,oidfop-mongodb), a PostgreSQL for HelixIAM and an nginx TLS proxy (oidfop-helix-proxy) run as containers. HelixIAM runs as the jar on the host, issuerhttps://helix-op.localtest.me:28943/realms/oidf; the proxy has that name as a network alias, so the suite and the browser use the same URLs. The proxy uses 32 KB response-header buffers, which the implicit and hybrid responses need (OpenID Connect flows). - Realm
oidf. Seeded through the admin API with the master realm's bootstrap service account: open dynamic client registration, static clients with the authorization code, refresh token and implicit grants and no consent screen, and a test useroidf-userwith a password, no MFA and every standard claim. - Driving. A Playwright driver creates each plan through the suite's API, runs every module, signs in on HelixIAM's pages, and uploads a screenshot for each review placeholder. A test idle for 90 seconds is stopped and recorded as such. The details are in the harness README.
Reproducing the run¶
The harness, its prerequisites and the commands are in e2e/oidf-conformance/README.md of the product repository.
The 13 plan configurations are in e2e/oidf-conformance/plans/; run.sh
takes the plan names to run (by default config, basic, formpost-basic and dynamic). One module can be re-run on its
own with driver/run-plan.mjs. export.sh writes the redacted logs into
docs/compliance/oidc-conformance/<date>/.
Redaction. export.sh removes every literal secret of the run, client secrets, passwords, access, refresh and
registration tokens, bearer and basic credentials, private JWK members and PEM private keys, and fails if a literal
secret survives. ID tokens stay: they are the signed evidence and carry only the fictitious test user's claims. The
suite's .sig files are dropped because they no longer verify after redaction.
Concerns¶
- Not certification. Formal OpenID Certification needs the OpenID Foundation (a fee, lower for members, and its review of the logs). Until then the accurate statement is that HelixIAM passes the suite's OP certification plans in a self-run, with the suite version and the date.
- W3 and W4 are choices, not gaps. If the Foundation's reviewers prefer the ID token without the scope claims when
an access token is issued, or an
acrfor a password sign-in, that is a conversation with them; both answers are within the specification.