It’s more important than ever to reduce the dependency on service accounts and ensure that automated actions are kicked off by the actual people requesting them, as opposed to shared credentials. This reduces the ways in which secrets can be inappropriately stored, it makes auditing much easier, and further centralises authentication and authorisation to IAM instead of application-specific API tokens and schemes.
In order for an automated action to be kicked off in a remote system, the system must trust that the request it's getting was properly authenticated; that a remote system is trusted beyond only the administrators involved enabling the connection, and can impersonate other users. Atlassian applications do this with OAuth through a framework called Application Links, available to developers as a Java API. Atlassian makes the source code available at packages.atlassian.com, which may assist with troubleshooting.
The Bitbucket team quietly released support for Application Links in Jenkins. Previously, Jenkins administrators would have to roll their own plugin extending Jenkins' OAuth Credentials framework: a challenging task, given the nuances of scopes and requirements of the OAuth Framework. Bitbucket can now kick off a job in Jenkins, without needing to store credentials in Bitbucket.
You can follow these steps for Jira too, and add Jira to Jenkins as a "Bitbucket server consumer". This should work for all Atlassian applications, as they share the same Application Links framework.
Ensure that Jenkins is using the same source for users as your Atlassian applications - whether it's Crowd, an LDAP server or a more exotic source. A version of the Bitbucket integration plugin Jenkins released after June 2020 must also be installed.
Inside Jenkins' Script Console, execute the following:
You will get a Base64-encoded public key string that you will need later on.
See the guide under "Create an Application".
Follow this guide to the point where it directs you to leave the Incoming Authentication box unchecked. As of writing, this was the last direction. Instead, check this box to go to the next page.
Now insert again the Consumer Key and Consumer Name that you set up on the previous page of the Application Link Wizard and in the Bitbucket Server Consumer setup in Jenkins. Finally, put in the public key string you obtained earlier and finish the setup.
Although you've created the connection, it will not work until you go to a special page in Jenkins to authorise Jira access to Jenkins and impersonate users.
At this point, it will either work or you will encounter obstacles. If you are redirected to Jenkins, you will receive a prompt like below, then "allow" to complete the trust establishment.
If there are any difficulties setting up this process, this guide may help.
There may be issues initially with Jenkins' CSRF protection: Jira may report "403 Forbidden" when attempting to request the OAuth Trust user authorisation URL from Jenkins, and Jenkins could report "hudson.security.csrf.CrumbFilter#doFilter: No valid crumb is included in request for /bitbucket/oauth/request-token by anonymous. Returning 403."
This can be worked around by temporarily disabling CSRF by adding the JVM system property "hudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true"
The root cause of this issue could be running Jira on the same machine as Jenkins, so Jenkins thinks the anonymous connection coming from localhost is too suspicious. (You may not encounter this issue).
After passing that hurdle, Jira may still refuse to offer the OAuth Trust user authorisation URL, as the hosts configured in the Application Link do not match the hosts in the exchange. One could try and follow various guides to configure the Nginx reverse proxy to correctly identify the hosts but bypassing the reverse proxy entirely is the solution suggested by Atlassian as a workaround. Atlassian provides a decent guide on this.
Adaptavist advises how application links can be used to access remote REST APIs.
Atlassian further explains here.