[Bug]: Code Analysis does not show alerts in PR if only custom cop warnings exist

[PSchwamborn-AINO]

AL-Go version

8.0 preview

Describe the issue

When I using custom code cops like the Linter Cop in Code Analysis the PR does not show any new warnings if they are only custom cop warnings. If there is at least one Microsoft Cop Warning the custom code cop warnings are shown corrrectly. But as soon as all Microsoft Cop Warnings are fixed the custom code cop warnings are also shown as fixed. Even if there was nothing changed.

Expected behavior

Custom Code Cop warnings should behave like Microsoft Cop Warnings.

Steps to reproduce

1. Set up a custom code cop such as Lintercop in the “customCodeCops” property.
2. Enable code analysis by setting “trackALAlertsInGitHub” to “true.”
3. Ensure that CI/CD has been run to initialize alerts in the main branch.
4. Create a PR and introduce a new Linter Cop alert -> No new alert is displayed in the PR ❌
5. Introduce a new Microsoft Cop warning by new Commit and Push -> A new alert is displayed in the PR and, surprisingly, the LinterCop alert is also displayed ✅.
6. Fix the Microsoft warning by new Commit and Push -> The Microsoft warning is shown as fixed, but the LinterCop warning is also shown as fixed ❌.

Additional context (logs, screenshots, etc.)

No response

[mazhelez]

Hi @PSchwamborn-AINO and thank you for reporting this issue.

To speed up the investigation, can you please provide logs from the PR build/CICD run when there was only the Linter Cop alert?

[PSchwamborn-AINO]

Hi @mazhelez, thanks for the reply.
Of course, here are the log files:

prbuild_logs_50087216027.zip

[spetersenms]

Hi @PSchwamborn-AINO, I have looked into this and tested a few different scenarios. I can get alerts to work with only LinterCop alerts and no AL alerts.
I also checked your logs and everything looks as it should, however I think I might have an idea why you are having issues:
There is a limitation to the alert comments on PRs from GitHubs side. The comments will only be shown if the alert location has actually been changed in the PR. That means, the specific line of code causing the alert needs to have an edit and show up with green in the 'changed files' of the PR, otherwise the comment will not be displayed.
I discovered this because I made a PR which activated Linter Cop, but did nothing else. The alerts from Linter Cop were in the artifacts, but not as comments on my PR.

Can you verify if the Linter Cop alerts for your PR actually correspond to a specific line change in your PR?

[PSchwamborn-AINO]

Hi @spetersenms, thank you for looking into it.
Yes, the alert is located on a line with a change.
I also tried again and now I can see the alert in the pull request. Strangely enough, I can only see the alert under "files changed" and not as a comment under “Conversation”.
Perhaps this is a problem with GitHub itself. If the alerts are visible in the files, that would be sufficient for us for now.