Skip to content

Conversation

@keithwillcode
Copy link
Contributor

@keithwillcode keithwillcode commented Dec 31, 2025

What does this PR do?

Reduces the default Playwright test timeout in CI from 60s to 30s to fail fast on slow tests, encouraging system improvements rather than waiting for slow operations.

Changes:

  • Overall test timeout: 60s → 30s in CI
  • Local timeout remains high (240s) for developer convenience
  • Step-level timeouts (navigation, action, expect) remain unchanged at 30s

This is a simplified version of the original PR - all 1-off timeout overrides and step-level timeout changes have been reverted per Keith's request.

Mandatory Tasks (DO NOT REMOVE)

  • I have self-reviewed the code (A decent size PR without self-review might be rejected).
  • I have updated the developer docs in /docs if this PR makes changes that would require a documentation change. N/A - configuration change only.
  • I confirm automated tests are in place that prove my fix is effective or that my feature works.

How should this be tested?

This change affects CI behavior only. To verify:

  1. Run the Playwright test suite in CI
  2. Tests taking longer than 30s total will now fail (previously 60s)
  3. This helps identify slow tests that need optimization

Checklist

  • My code follows the style guidelines of this project
  • I haven't checked if my changes generate no new warnings

Human Review Checklist

  • Verify 30s overall test timeout is acceptable for CI (reduced from 60s)
  • Note: Step-level timeouts remain at 30s (unchanged from main)
  • Consider if any tests legitimately need more than 30s and may need individual test.setTimeout() overrides added later

Link to Devin run: https://app.devin.ai/sessions/8df4c87ad80846c9b3d358f520f174ed
Requested by: keith@cal.com (@keithwillcode)

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@vercel
Copy link

vercel bot commented Dec 31, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Review Updated (UTC)
cal Ignored Ignored Dec 31, 2025 4:09pm
cal-companion Ignored Ignored Preview Dec 31, 2025 4:09pm
cal-eu Ignored Ignored Dec 31, 2025 4:09pm

@github-actions
Copy link
Contributor

github-actions bot commented Dec 31, 2025

E2E results are ready!

@devin-ai-integration devin-ai-integration bot changed the title fix: reduce Playwright timeouts to fail fast (2s step-level, 15s test) fix: reduce Playwright timeouts to fail fast (5s step-level, 15s test) Dec 31, 2025
Co-Authored-By: keith@cal.com <keithwillcode@gmail.com>
@devin-ai-integration devin-ai-integration bot force-pushed the devin/1767190878-reduce-playwright-timeouts branch from 6962bcb to c140df0 Compare December 31, 2025 15:21
@devin-ai-integration devin-ai-integration bot changed the title fix: reduce Playwright timeouts to fail fast (5s step-level, 15s test) fix: reduce default Playwright test timeout from 60s to 30s in CI Dec 31, 2025
@keithwillcode keithwillcode self-assigned this Dec 31, 2025
@keithwillcode keithwillcode added this to the v6.1 milestone Dec 31, 2025
@keithwillcode keithwillcode marked this pull request as ready for review December 31, 2025 15:28
@keithwillcode keithwillcode enabled auto-merge (squash) December 31, 2025 15:29
@graphite-app graphite-app bot added foundation core area: core, team members only labels Dec 31, 2025
@graphite-app graphite-app bot requested a review from a team December 31, 2025 15:29
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="playwright.config.ts">

<violation number="1" location="playwright.config.ts:21">
P2: The comment above this line states the test timeout &quot;should be much higher than sum of expect and navigation timeouts&quot;, but the new value (30s) is now equal to (not higher than) the individual step timeouts (navigation: 30s, expect: 30s). Consider updating the comment to reflect the new intentional design choice of failing fast on slow tests rather than accommodating multiple slow operations.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 1 file (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="playwright.config.ts">

<violation number="1" location="playwright.config.ts:16">
P1: The actual changes contradict the PR description. The description states &quot;Step-level timeouts (navigation, action, expect) remain unchanged at 30s&quot;, but this line reduces navigation timeout from 30s to 10s in CI. If the intent is to keep step-level timeouts at 30s as stated, this change should be reverted to `30000`. If 10s is intentional, the PR description needs to be updated.</violation>

<violation number="2" location="playwright.config.ts:17">
P1: Same issue as above - this contradicts the PR description which states &quot;Step-level timeouts (navigation, action, expect) remain unchanged at 30s&quot;, but this reduces expect timeout from 30s to 10s. Additionally, the comment above states test timeout should be &quot;much higher than sum of expect and navigation timeouts&quot; - with test timeout at 30s and this at 10s, the ratio is tighter than before.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@keithwillcode keithwillcode merged commit c959fff into main Dec 31, 2025
45 checks passed
@keithwillcode keithwillcode deleted the devin/1767190878-reduce-playwright-timeouts branch December 31, 2025 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants