Setup and Configuration Issues with oxide-rfd

[ChielTimmermans]

Setup and Configuration Issues with oxide-rfd

Summary

While attempting to set up oxide-rfd, we encountered multiple significant issues that hindered deployment and made the system difficult to configure properly. This issue documents all problems encountered to help improve the project's usability and documentation.

Environment

• Deployment: Docker containers
• Build mode: Production
• Repository fork: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd

Issues

1. Missing Docker Container Support

Problem: The project does not provide official Docker container configurations, making deployment more difficult than necessary.

Impact: Users need to create their own Dockerfiles from scratch, which is time-consuming and error-prone.

Our Solution: We created custom Dockerfiles for each component:

• API: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd/-/blob/main/api.Dockerfile
• Processor: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd/-/blob/main/processor.Dockerfile
• Site: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd/-/blob/main/site.Dockerfile
• Migrator: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd/-/blob/main/migrator.Dockerfile

Additional Issue: The documentation makes no mention of the database migrator component, which is essential for setting up and maintaining the database schema. We discovered this requirement through trial and error.

Recommendation:

• Provide official Docker support with example compose files and Dockerfiles for all components
• Explicitly document the migrator component and its role in the setup process
• Include migration instructions in the deployment documentation

---

2. Outdated API Configuration Example

Problem: The API configuration example in the documentation is outdated and uses deprecated key types.

Details:

• Old format: type = local
• New format requires: type = local_signer and type = local_verifier

Impact: Users following the documentation will encounter configuration errors and need to debug the correct format themselves.

Recommendation: Update all configuration examples to reflect the current API requirements.

---

3. Configuration Parsing/Merging Issues

Problem: The configuration parsing and merging logic appears to be buggy, forcing users to define the entire configuration structure as secrets rather than splitting between config files and environment variables/secrets.

Expected Behavior: Users should be able to:

• Define non-sensitive configuration in config files
• Override sensitive values with environment variables/secrets
• Have the system properly merge these sources

Actual Behavior: The merging doesn't work reliably, forcing parts of configuration into a single source (typically secrets).

Impact: This violates configuration best practices and makes the system harder to manage across different environments.

---

4. Environment Variable Parsing for Nested Configuration

Problem: The configuration system does not properly parse nested environment variables.

Attempted Formats:

• Single underscore: AUTH_GITHUB_TOKEN
• Double underscore: AUTH__GITHUB__TOKEN

Result: Neither format worked for nested configuration values.

Expected Behavior: One of these standard formats should work for nested config (e.g., AUTH__GITHUB__TOKEN for auth.github.token).

Impact: Users cannot use environment variables for configuration override, which is a standard practice in containerized deployments.

---

5. First Login Process Issues

Problem: The initial authentication setup is problematic in production builds.

Details:

• Documentation suggests running the program with a build-time flag
• This approach doesn't work when building in production mode
• There appears to be no runtime flag alternative

Note: This could potentially be user error, but the current documentation and tooling don't make the correct approach clear.

Recommendation:

• Provide a runtime flag for initial authentication setup
• Clearly document the first-login process for production deployments
• Consider alternative bootstrap authentication methods

---

6. CLI Authentication Flow Broken

Problem: The rfd-cli auth login github command exits immediately without waiting for user verification.

Expected Behavior:

1. Command initiates GitHub OAuth flow
2. User completes authentication in browser
3. CLI waits for callback/verification
4. CLI confirms successful authentication

Actual Behavior:

1. Command starts
2. Command exits immediately
3. No authentication is completed

Impact: Users cannot authenticate via the CLI, blocking command-line workflows.

---

7. RFD Scanner/Processor Not Creating New RFDs

Problem: The scanner and processor detect new RFDs in pull requests and create job entries, but the RFDs are not actually created or displayed in the frontend.

Observed Behavior:

• New RFDs with PRs are detected
• Jobs appear in the database jobs table
• Logging shows processing activity
• However: No RFDs are created in the database
• Result: Nothing appears in the frontend

Impact: The core functionality of processing new RFDs is broken, making the system unusable for its primary purpose.

Debugging Information Needed:

• Are there error logs that aren't being surfaced?
• Is there a database schema mismatch?
• Are permissions issues preventing RFD creation?

---

8. Token Refresh Broken in Site

Problem: When authentication tokens expire, the site does not attempt to refresh them automatically.

Expected Behavior:

• User's token expires during session
• Site detects expiration
• Site automatically requests new token
• User session continues seamlessly

Actual Behavior:

• User's token expires
• Site does not refresh the token
• User can only see public RFDs
• User must manually log out and log back in to restore access

Impact: Poor user experience requiring frequent re-authentication. This breaks standard OAuth token refresh flows.

Recommendation: Implement proper token refresh logic before expiration or on 401 responses.

---

9. GitHub Webhook Secret Configuration Missing

Problem: GitHub webhooks are failing with an error indicating that a secret needs to be set, but there is no documentation on how or where to configure this secret.

Error Received: Error message about needing to set a webhook secret (exact error message would be helpful if available).

Documentation Gap:

• No mention of webhook secret configuration in setup documentation
• No example configuration showing where to set the secret
• No guidance on how to generate or format the secret

Expected Information:

• Configuration parameter name for the webhook secret
• Where to set it (config file, environment variable, etc.)
• How to generate an appropriate secret value
• How to configure the same secret in GitHub's webhook settings

Impact: GitHub integration is non-functional, preventing automatic RFD updates from repository events.

Questions:

• Is this related to the configuration parsing issues (Issue 3, 4)?
• Should this be set as an environment variable?
• Is there a specific format or length requirement for the secret?

---

Summary of Critical Issues

The most blocking issues are:

1. RFD Scanner/Processor not creating RFDs (Issue 7) - Core functionality broken
2. GitHub webhooks broken (Issue 9) - GitHub integration non-functional
3. CLI authentication broken (Issue 6) - Cannot use CLI tools
4. Token refresh not working (Issue 8) - Poor user experience
5. Configuration system issues (Issues 3, 4) - Makes deployment difficult

Request for Maintainers

We would appreciate:

• Guidance on whether we've misconfigured something
• Timeline for fixes if these are known issues
• Documentation updates for workarounds if available
• Consideration of our Docker configurations as a starting point for official support

We're happy to contribute fixes for some of these issues if given guidance on the preferred approach.

Additional Context

Our fork with Docker configurations and workarounds: https://gitlab.com/digilab.overheid.nl/ecosystem/oxide-rfd

Thank you for maintaining this project. Despite these issues, we see significant value in the RFD system and hope to get it fully operational.

[augustuswm]

Greatly appreciate you putting this all together and great to see others trying this out. To preface all of the below, some of them are a bit difficult to help with without some more information or seeing what the service is logging. I am happy to help dive into logs and troubleshoot issues if you like. Feel free to ping me.

1. Happy to accept PRs that assist in making it easier to run the rfd-api in as a Docker container. We do not deploy it as a container so creating or maintaining support for Docker containers is not a priority for us.

2. Thanks for the catch. This definitely needs an update now that key components have been split.

3. We are looking at some possible changes around how configuration is handled and read. Specifically I want to use a layered file approach, but one that is not necessarily focused on separation between variables and secrets. I am not positive that we will go the route of environment variables though, and may instead defer that work to the packaging / deploying technology. (in the case of Docker this could be something like a processor that reads ENV variables and creates the config file.

4. (see 3)

5. This should be performed via a mapping file that contains initial groups and JIT rules for assigning groups and/or permissions to users. See: https://github.com/oxidecomputer/rfd-api/blob/main/rfd-api/mappers.example.toml for an example. The local-dev flag should expressly not be used for initial user creation and is intentionally a build flag due to that. This flag enables an insecure endpoint that is explicitly there to assist in local development and testing purposes only.

6. I would need to see additional information here, but this is likely an issue with the rfd-api service configuration. The server is likely failed to return a valid authorization initiation response to the CLI. That said this should definitely have an improved error message.

7. I would verify that the processed column is actually being marked as t to ensure that the records did not fail to process part way through. If they are not then there should be an error message related to one of the actions that have been defined in actions setting in https://github.com/oxidecomputer/rfd-api/blob/main/rfd-processor/config.example.toml. The rfd-processor service has its own log file and that is likely the place you want to start looking for errors. The rfd-api will only write jobs, and generally it is not intended to do any processing of jobs.

8. Currently this is intentional. We do not refresh tokens and instead allow them to expire, requiring the user to log in again. If a user's session cookie is expiring before their API token expires, then there is likely a mismatch between the lifetime of the API token and the session cookie. I do not recall if the frontend automatically tries to synchronize these times. If it doesn't then it should.

9. If you mean within the rfd-api service, this is definitely a missing piece of documentation. Due to a limitation in how the rfd-api service is configured there is one environment variable that does need to be set. Specifically GITHUB_WEBHOOK_KEY should be set to the secret that you have defined in the GitHub App.