The CA Agile Central VCS Connector for Bitbucket Server and Bitbucket Cloud posts information about Bitbucket repository commits to CA Agile Central, relating those commits to CA Agile Central changesets and artifacts if sufficient information is contained in the VCS commit messages to identify the relevant artifacts. The connector will also update the state of relevant CA Agile Central artifacts, specifically defect state or user story schedule state when commit messages contain appropriate syntax, for example US123 In-Progress. The CA Agile Central VCS Connector for Bitbucket is classified as a one-way and one-time mechanism. Information in Bitbucket is never altered; information is only written in Agile Central and no duplication of data is attempted or allowed. The CA Agile Central VCS Connector for Bitbucket consists of software that you run on your platform according to your desired schedule. The configuration of the connector is policy-based, meaning that you do not need to provide a separate configuration file for each Bitbucket repository for which you want the connector to operate. The policy-based nature allows you to scope to all or a subset of repositories accessible to a Bitbucket user or organization.
The Agile Central VCS Connector as of version 2.0.3 includes functionality that detects the creation of Bitbucket pull request items and posts corresponding pull request items in Agile Central when the configuration option for this is present in the target configuration file. Currently, the pull request functionality is only available on User Stories. To show pull requests in Agile Central as Third Party Context, add the PullRequests value set to True. The AC artifact formatted ID (ex. US1234) can be mentioned in either the pull request title or in of one of the commit messages that is associated to the pull request in order to be processed by the connector. The connector will include open as well as a merged pull requests.
This connector uses Bitbucket REST API, and will operate with both Bitbucket Server and Bitbucket Cloud. Depending on if you are using Bitbucket Server or Bitbucket Cloud, there will be minor variations in the configuration file items.
This guide includes:
- Software Requirements
- Connector Download
- Appendix A
- Revision History
- Linux, Windows and Mac Platforms are supported.
A valid subscription and credentials to CA Agile Central.
An active Bitbucket account. Bitbucket versions 5.0, 5.1, and 5.2 are supported. Other 5.x versions may work.
Supported on Python 3.5.x or 3.6.x (we recommend using the 64-bit version on Windows). You can retrieve the package installer from www.python.org.
Windows-only additional requirements:
The pywin32 package must be downloaded and installed subsequent to Python 3.5 or 3.6 installation.
The pywin32 package is available from: https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/pywin32-221.win-amd64-py3.5.exe/download and https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/pywin32-221.win-amd64-py3.6.exe/download.
To download the connector, follow the steps here.
You can install the connector several ways. There can be multiple install locations on the same machine that can run in parallel which determine the hardware required. We recommend the following guidelines:
- A computing platform on which your connector will run (Windows, Linux, Mac)
- Base hardware with a quad-core processor set and at least 8 GB RAM
- No more than 20–25 configuration files per install location
- Adequate disk storage capacity to store log files generated by the execution of the connector; we recommend 1 GB of free disk space
- If you add more install locations running in parallel, each installed location consumes up to 200 MB of RAM when running and less than 50 MB when idle
- 200–250 MB of space per install location is recommended since the required hard drive space is minimal and needs to scale only with log files
**Please navigate to the bottom of this document (Downloads section) for a sample file of a config with all the options available for this connector. The file is called: "a sample file with explanatory notations.txt"
- Once you have Jenkins and Python installed - and for Windows users only, the win32com module installed (information for the previously mentioned can be found in the "Software Requirements" section above), install the following Python packages:
pip3 install requests==2.18.4 pip3 install pyral==1.4.2 pip3 install PyYAML==3.12 pip3 install cryptography==1.7.1
- Unpack vcsconn-*.zip. Change your working directory (cd) to a directory where you want to install the connector. Unzip vcsconn-*.zip (or use a suitable program that can unzip a .zip file):
cd vcsconn-2.0.3 ls -laR # observe the unpacked contents, or use dir on Windows vcsconn # vcsconn module root directory vcs2ac # connector initiation script, this is what you will run config # holds any configuration files used with this connector sample_bitbucket_server.yml# a sample configuration to use as a base reference for Bitbucket Server sample_bitbucket_cloud.yml # a sample configuration to use as a base reference for Bitbucket Cloud sample_user_map.yml # a sample text file for user mapping log # hold log file and time file per configuration README_BITBUCKET.txt # this file User-Mapping-Strategies.txt # details Bitbucket user to AgileCentral username mapping strategies
- Start simple.
- Verify that your target Workspace has BuildandChangsetEnabled set to true. Your CA Agile Central Workspace Administrator will need to enable this for your target workspace. If it is not enabled, the connector will work, but you will not be able to see any of the changeset or change information that is associated with your CA Agile Central work items (story, defect, task). The workspace administrator must edit the workspace and do the following:
- Locate the configuration subdirectory.
- Copy the sample_bitbucket_(server or cloud).yml file to a file named suitably for your environment for example, cp sample_bitbucket_server.yml to product_x.yml (or some other suitably named file that has a .yml suffix).
- Edit your product_x.yml file. For example, change the sample values for credentials, workspace, project, token, and so on to values that are relevant and valid for your environment.
- Decide on the appropriate SecurityLevel.
- Decide if you want to update state of CA Agile Central artifacts based on commit messages.
- Decide if you want to map a Bitbucket committer to a valid Agile Central user and choose one of the three user mapping methods: Passthrough, FileBasedUserNameLookup, or UserNameDomainAugmentLookup. See the User-Mapping-Strategies.txt file in the main connector directory for further detail. Note: The Bitbucket Cloud field Full Name and the BitBucket Server Name field will be used in the user_map.txt file for the first column.
- Decide on whether the repositories you want processed are repositories you own (use UserSlug) or are repositories owned by a team of which you are a member (use TeamSlug).
*See Appendix A on configuration file syntax.
You can choose to perform manual or scheduled operation of this connector.
Using a terminal window or console: cd to the installation root directory, for example: /opt/local/sw/vcsconn-1.2.4
python3 vcs2ac product_x.yml
This software requires that the configuration file reside in the configuration subdirectory. You specify the name of the file on the command line. Do not specify the subdirectory in the command line argument.
Use either cron, launchctl, or Windows Task Scheduler. Make sure the environment in effect when running this software has an appropriate environment set so that you can run:
python3 $VCS/vcs2ac your_config_file_name.yml
where $VCS is the reference to an environment variable containing the fully qualified path to the directory where the software is installed. For example, if you unzipped the package in /opt/local/sw, then your VCS would be set like this:
In normal operation, the connector writes a time file (in the log directory) whose name is based on the configuration file name. Example: If the configuration file name is product_x.yml, then the associated time file name would be product_x_time.file. The content of the time file is a single line containing a human readable date/time stamp value in the format YYYY-MM-DD hh:mm:ss Z. The value represents the timestamp of the last commit reflected in Agile Central. When the connector is run a subsequent time, it consults the time file to determine which jobs need to be considered for the current run by only processing the commits whose start time is greater than or equal than the time file value. It is possible to set the time file value artificially (using ISO format, such as 2017-05-12 11:37:55 Z), so that you can go back in time and pick up commits from some arbitrary point in the past. The connector does not duplicate commits so you do not have to worry about duplicated information getting posted to Agile Central. In the absence of a time file, the connector pulls commits starting at a point three days prior to the current time.
The connector always writes a log file named based on the configuration file name. The log file is written into the log subdirectory under the base installation directory. You can control the extent of logging by specifying the level in the configuration file. Within the configuration file, you can set the LogLevel which determines the amount of logging information written to the log file. Under the Service section, modify the LogLevel setting (valid values are ERROR, WARN, INFO, DEBUG). When you set the LogLevel to DEBUG, you will get the full range of logging messages that can be helpful in pinpointing where a problem occurred and what information was being processed at the time of the anomaly.
It can be very helpful to run the connector in Preview mode when setting things up for the first time. This allows you to get the connections to Agile Central and Bitbucket to initialize and validate correctly without posting any changesets.
*Issue: FATAL: VCSConnectorRunner.run - BitbucketConnection.connect bitbucket_connection.py(106) - Unable to connect to Bitbucket Server at https://bitbucket.xxx.xxx.com:443/rest/api/1.0/projects/ISSP. 'key'
Answer: Verify that username and password for Bitbucket are correct.
*Issue: Error Seen On Windows 10 64 Bit. FATAL: DLL Load Failed: The Specified Module Could Not Be Found.
Answer: Change environment variable path from ...Python/Scripts to ...Python/DLLs.
*Issue: Pull Requests are not transferring to Agile Central.
Answer: Currently, pull request functionality is only available for User Stories in Agile Central. If you are not seeing Pull Requests in User Stories, please verify the following line is in the service section config file:
PullRequests : True # Default is False, when True VSTS PullRequests are detected and reflected in Agile Central
*Issue: Error seen when using * in the include statement to pull all repos.
FATAL: VCSConnectorRunner.run - VCSConnectorRunner.getConfiguration vcs_connector_runner.py(288) - Unable to parse config/vsts-cloud.yml successfully, while scanning an alias in "<unicode string>", line 25, column 26: Include : * ^ expected alphabetic or numeric character, but found '\n' in "<unicode string>", line 25, column 27: Include : * ^
Answer: Escape the * in the include statement with quotes.
Repository: Include : "*"
When credentials, including ProxyUsername and ProxyPassword, in the configuration file are no longer in clear text, SecurityLevel settings can only upgraded, but not downgraded. For example, a configuration file with encrypted credentials cannot be used with SecurityLevel reset from Encrypted to Encoded. If users want to downgrade security level, they have to start with a configuration file where credentials are in clear text.
Appendix A - Configuration File Editing
The Agile Central VCS connector for Bitbucket uses a text file in the YAML format. For complete information, consult the web page at https://www.yaml.com or any other website on the subject of YAML. Please see the Download Samples section at the bottom of this page for a sample configuration file with explanatory notes.
For brevity, this document mentions several of the most relevant syntax items and covers the three sections of a valid YAML configuration that can be used with the connector.
- Use a text editor (not MS Word or Google Doc) to edit the file.
- Never use a tab character in the file. YAML does not allow or recognize tab characters.
- Save the file in UTF-8 format.
- Use a monospace font.
- Be consistent with the number of spaces you use to indent.
- On a line, the first occurrence of a non-quoted # character indicates a comment, the # character and all characters to the right are ignored in processing.
- Keep the sections in the same order as is present in the sample.yml file.
- Be aware that the colon character : is significant, it separates a key from the value.
- Be aware that the dash character - is significant, it denotes the start of a list which may have one or more key value pairs that constitute the list item.
- You usually do not have to quote values if they have spaces in them; you will have to quote the value if it contains an embedded # character.
Skeleton of the template_config.yml file
Bitbucket: AgileCentral: ... # several key value pairs are relevant for this section Bitbucket: ... # several key value pairs are relevant for this section Service: ... # a few key value pairs relevant for the overall operation # of the connector appear in this section
The AgileCentral section specifies values to use to obtain a connection with Agile Central. The Bitbucket section specifies values to use to obtain a connection with Bitbucket and specify the policies governing what repositories in Bitbucket get processed to result in posting of changesets to Agile Central. The Service section controls some aspects of the connector behavior on an overall basis.
*The required Project line in the CA Agile Central section is used for for connection purposes; commits will still show in CA Agile Central based on the workspace scope.
- 2.0.3-Master --- 9-Feb-2018
- Fixed name of StateExtractor class in sample config file
- Fixed assignment of Changeset URI for Bitbucket Server
- 2.0.0-Master --- 21-Dec-2017
- Unified all newer policy-based VCS connectors into one distribution package
- 1.3.2-Master --- 7-Dec-2017
- Added support for pull requests
- 1.3.1-Master --- 4-Dec-2017
- Transition from author to committer for Bitbucket Server
- Added support for appending committer name to commit message
- 1.2.4-Master --- 17-Nov-2017
- Connector rewritten in Python
- Addition of Policy-based feature
- Support added for Bitbucket Cloud