Connecting to Git / GitHub

The Magic Suite file system can use a chosen Git repository and branch as a virtual read-only folder.

For credentials, when using a GitHub URL in the Connection's "Url" field, you need a Personal Access Token (PAT). GitHub offers two types, which it calls a 'fine-grained PAT' and a 'classic token'. Either one works.

Note: GitHub does not accept an account password. GitHub withdrew password support for Git access on 13 August 2021, so a Connection that uses your GitHub password will always show as invalid, however carefully it is entered. Use a PAT instead.

Git servers other than GitHub are a different matter. If the Connection points at a Git server that still accepts a user name and password, you can use those in the Username and Password fields as normal.

More information about creating a PAT can be found on GitHub here.

To set up Magic Suite apps to be able to use Git / GitHub as a file system Connection:

• If your repo is private:
• For a GitHub repository, create a Personal Access Token (PAT) and note it down. The sections below explain how to create one
• For any other Git server, note the user name and password that it expects
• Decide which branch you want to use (typically, 'main')

Then in the Admin app, under Connections:

1. Click the Create button
2. Enter a name and optionally, a description. The name specified here will be the name used in the file system
3. From the Type drop-down, choose a Connection type of Git / GitHub
4. In the URL field, enter the Git repository URL e.g. https://github.com/company/Repository
5. Is your repo private?
1. Yes, and it is on GitHub: leave the Username field blank, and paste your PAT into the Password field
2. Yes, and it is on another Git server: enter the user name and password that server expects
3. No: leave the Username and Password fields empty
6. In the mandatory Configuration field, paste this and change the values: { "FileSystemMountPoint": "/MyGitRepo", "Branch": "mybranch" } e.g.
• Change "/MyGitRepo" to any name - this is what should be used in macros, such as [File.xxx:] macros
• Change "mybranch" to the branch of the repository, e.g. typically just "main"
• Optionally, add "RepositoryPullIntervalSeconds": 60 to control how frequently Magic Suite refreshes the repository from the remote - see Repository Pull Interval below
7. Click Save
8. Go to the Files browser in Report Magic and you should notice that a GitHub icon appears, and the repository will be cloned into the correct file structure
9. Go to the Connections page (in ReportMagic it's Admin -> Connections, or in the Admin app just the Connections menu) and a green checkbox should appear in the table if the details you have entered are valid

Once this process is completed, you should then be able to access files and folders (but not write to) in the repository in your ReportMagic reports as normal, e.g. by using various [File.xxx:] macros.

In the files browser, the name of the 'drive' is the name of the Connection. However, in macros, the path you should use is the FileSystemMountPoint.

Using Personal Access Tokens (PATs) to Access Private GitHub Repositories

You can access a private repository (e.g. with a PAT) if you are the owner, or have been invited by someone else to be a collaborator.

In order to work with Magic Suite apps (i.e. show whether the Connection is valid on the Connections page, and to be able to read the repository contents), there are a few things you should do when using GitHub, as described below.

Note that it is not necessary to enter a Username in the Connection when using a PAT. GitHub does require a user name to be sent to it, but Magic Suite recognises a PAT and supplies one for you, so you can leave the Username field empty.

When using a 'Fine-Grained' PAT

To create one:

• Go to GitHub and log in
• Click your profile picture in the top-right of the page, then click Settings
• On the page that appears, scroll down and choose Developer settings from the left-hand menu
• On the page that appears, choose Personal access tokens from the left-hand menu
• Select Fine-grained tokens from the drop-down, rather than Tokens (classic)
• Click the green Generate new token button that appears
• Give the token a name and an optional description, and choose the expiration date
• Under the Repository access section, choose either All repositories, or Only select repositories
• Under the Permissions section, expand the Repository permissions area
• Scroll down to Contents, and in the Access drop-down, choose Read-only (you can choose Read and write, but in Magic Suite we only use read only access so it is currently unnecessary)
• Scroll down and press the green Generate token button
• A pop-up appears - click the green Generate token button (yes, it makes you click such a button twice!)
• On the next page, copy the token details
• Use this token in the Password field of the Connection create/edit dialog
• Save the Connection - the status icon in the Connections table will become a green check if everything has worked
• You may also click the status icon to check its validity again
• Go to the Files browser in Report Magic and check that the repository has been cloned

When Using a 'Classic' PAT

The steps are almost identical to the above, except:

• When creating the token, choose Tokens (classic) when choosing the token type
• Instead of a name and description, you are asked to complete a mandatory Note field, that describes what the token will be used for
• There is no Repository access and Permissions sections
• Instead, in the Select scopes section, select the repo check-box, in order to give the required permissions

Repository Pull Interval

The optional RepositoryPullIntervalSeconds configuration property controls the minimum number of seconds that must elapse between pulls from the remote Git repository. This prevents a network fetch from being triggered on every individual file access, which is particularly important when multiple Schedules run simultaneously against the same repository.

• Default: 60 seconds
• Valid range: 1 to 86400 (1 second to 24 hours)
• Within the cooldown window, file accesses use the already-checked-out local working tree without contacting the remote
• Once the interval has elapsed, the next access will trigger a pull to fetch any new commits from the remote

Example configuration using a 5-minute pull interval:

{ "FileSystemMountPoint": "/MyGitRepo", "Branch": "main", "RepositoryPullIntervalSeconds": 300 }

If your repository changes infrequently and you want to reduce network traffic, consider increasing this value. If you need near-real-time access to the latest commits, reduce it towards 1. The default of 60 seconds is suitable for most use cases.