Agent FAQ and Troubleshooting

FAQ

Q: How do I exclude specific files from a watch folder from being uploaded for an agent that uploads files and runs flows?

A: To exclude specific files from being uploaded, you can use the fp function to specify which files should be included. For example, to exclude files that contain the string "exclude" in the filename, you can use the following code:

def fp_res(x: str):
    x = parse.unquote(x)

    # files excluding temp files
    files_to_capture = [filename in glob.glob(os.path.join(watch_dir, pattern), recursive=True) if not 'exclude' in filename]

    return files_to_capture

return fp_res

Q: How do I capture a file that is periodically overwritten by an instrument?

A: File watcher Agents automatically capture modified files, which they then upload for processing in flows. You can implement logic within the flow or agent to handle the intended behavior, depending on whether the instrument appends to or overwrites the file.

Q: Can I access environment secrets from the Agent?

A: Yes, environment secrets can be accessed from the Agent by using the get_secret method.

Q: Can I access an environment variable from a cron Agent? For example, can I access the input_path variable configured upon installation?

A: Yes, environment variables can be accessed from a cron Agent through the kwargs dictionary. For example, to access the input_path variable, you can use the following code:

# .get method for dictionries is used to handle the case where the variable may not be present without erroring out
# note: assumes DEFAULT_PATH is configured in the agent
watch_directory = Path(kwargs.get("vars", {}).get("input_path", DEFAULT_PATH))

Q: Can Windows Agents be installed headlessly?

A: Yes - Agent installers (v4.10+) can be installed headlessly.

Requirements

The following files are needed:

• Agent installer (EXE, not MSI) with a minimum core Agent version of 4.10
• Configuration file (YAML); examples of this can be found in the directory specified by the installer when not installed headlessly. Start with an existing YAML configuration and modify as follows:
  • Include:
    • environment (required)
    • name (required)
    • variables
    • tagParams (if applicable)
  • Exclude:
    • id
    • inferredData (any)
    • startTime
    • Version

A Windows batch script, such as the one shown below, saved to a .bat file:

@ECHO OFF

:: Agent name (used for defining install path) defined by passed argument:
set agentname=%1

:: Set install path assuming agentname was passed:
set installpath=C:\Program Files\Ganymede\%agentname%\

:: If agentname was not passed, remove double filesep:
IF "%~1" == "" (
	set installpath=C:\Program Files\Ganymede\
)
ECHO Step 1: Creating install path: %installpath%
mkdir "%installpath%"

ECHO Step 2: Copying exe file
:: Note: %~dp0 is the path of the script (needed when running as admin)
:: Note: /v verifies the copied file, /y forces a replace if file with same name exists
copy "%~dp0agent.exe" "%installpath%" /v /y

ECHO Step 3: Copying config file
copy "%~dp0connection.yaml" "%installpath%" /v /y

ECHO Step 4: Running installer
start "%installpath%" "%installpath%agent.exe"
:: Uncomment next line for testing purposes if needed
:: PAUSE

Execution steps

1. Rename agent installer “agent.exe”
2. Name configuration file “connection.yaml” (if it has a different name)
3. If desired, zip up a folder containing the agent installer, connection.yaml, and Windows batch script to facilitate sharing.
4. On the target Windows machine, run the batch script via command prompt or Power Shell (as Administrator).
   1. To do this manually, unzip the installation package if necessary, and run command prompt or PowerShell (as Administrator).
   2. In command prompt or PowerShell, run the batch script via command line, optionally supplying the agent name as an argument
      1. (e.g. "C:\user\agent_install.bat HamiltonAgent")
      2. If argument is not supplied (e.g. when batch script is run from Windows directly), the install path will default to "C:\Program Files\Ganymede"
   3. To do this via a script, create a task in Task Scheduler pointing to the Windows batch script.
      1. Name the task, and check Run with highest privileges.
      2. In the Actions tab, add a new action pointing to your batch script.
         1. In the "Add Arguments" field, optionally provide the agent name. If argument is not supplied, the install path will default to "C:\Program Files\Ganymede"
      3. In the Settings tab, ensure that the task is configured to run without user intervention.
      4. In the Triggers tab, set an appropriate condition to trigger the task.

Once set, invoke the task using schtasks:

schtasks /run /tn "TaskName"

Q: Are there any command-line scripts (e.g. - .vbs or .bat) that need to run for Windows Agents to work?

A: No, there are no command-line scripts that need to execute for the Windows Agent.

Q: Are there any known application dependencies or pre-requisites for the Windows Agent (e.g. - .NET, KBs, Redist)?

A: No, the full set of Agent requirements can be found on the Agent System Requirements page.

Troubleshooting

Agent build

Issue: The build for a "Watch for files locally then run flow" Agent keeps failing, with no Windows executable or Linux binary created.

Agents that watch for files locally then run Flow require file inputs to be passed to the Flow for execution.

1. Ensure that the Flow being triggered by the Agent has at least one Node for ingesting files.
2. Ensure that Agent creation occurs after Nodes for input files have been created; if necessary, create a new Agent after the Flow has input Nodes; to see that this is the case, you should see dropdown selectors for specifying glob patterns to associate with the input Nodes of the Flow corresponding to the new Agent.

Agent installation

Issue: I am unable to install an Agent Connection on Windows; the installer is failing at the final step.

Ensure that you are running the installer as an administrator.

Check for existing installations that might conflict with the current installation:

• Open Task Manager and navigate to the Services tab.
• Look for any service starting with "GanymedeAgent-" and stop it by right-clicking and selecting "Stop."
• Open "Add or Remove Programs" and uninstall any existing Ganymede Agent installations.
• Try installing the Agent again.

Issue: I've installed an Agent Connection, but do not see the Connection in logs.

This is likely a network issue, which can be confirmed by testing connectivity.

Another way to confirm this is to see

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate`

in the Connection logs file found locally on the computer. These logs can be found in the directory that the Agent Connection is installed in.

To resolve, ensure that outbound port 443 communication is available to the web addresses mentioned on the Agent System Requirements page.

Issue: The Agent Connection is not actively running. How do I check?

On Windows:

• Open Task Manager and navigate to the Services tab.
• Look for the Connection of interest (a service with a name starting with "GanymedeAgent-") and verify that its status is "Running."
• If the service is not running, right-click on it and select "Start." Connections are configured to start upon reboot by default.

On Linux:

• Open a terminal and run the following command to check the Agent's status:

  sudo systemctl status GanymedeAgent-<agent_name>

  If the Agent is installed but not running, you can start it by running:

  sudo systemctl start GanymedeAgent-<agent_name>

Agent execution

Issue: Agent Connection is showing as disconnected after running for a while

On Windows:

• Ensure that the PC where the Agent is installed is not asleep. PCs running cron Agents should be configured to never sleep:
  • Open the Control Panel and navigate to "Hardware and Sound" -> "Power Options."
  • Select "Change when the computer sleeps" and set "Put the computer to sleep" to "Never."
• Verify that the system time is properly synced:
  • Right-click on the time in the bottom right corner of the screen and select "Adjust date/time."
  • Ensure that "Set time automatically" is enabled, and click "Sync now."

Issue: The flow is failing to trigger when files are being dropped

On Windows:

Look at the logs for the agent in Ganymede App. If you see the JWT error below, click on the system clock in the lower right corner of the screen and select "Adjust date/time". Ensure that the "Set time automatically" option is enabled, and click "Sync now".

_thread.ExceptHookArgs(exc_type=<class 'google.auth.exceptions.RefreshError'>, exc_value=RefreshError('invalid_grant: Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.'})

• Right-click on the system clock and select "Adjust date/time."
• Ensure that "Set time automatically" is enabled, and click "Sync now."

Verify that the watch folder is correctly specified during Agent installation.

-v "input_path=C:/Users/<username>/Desktop/watch_folder"

Take care when copying and pasting commands

Be careful when copying and pasting commands. The quote character " can sometimes be replaced with left or right quotes (“ or ”), which may cause issues. Always ensure that the straight quote character " is used.

Issue: All files are present in the watched directory, but the flow does not trigger. The logs show files being collected in two different waiting groups.

A: Unquote the string referenced within the fp function, which specifies which filename patterns the Agent should watch for. An example is shown below:

def fp(watch_dir: str, parent_dir: str, pattern: str) -> Callable[[str], bool]:
    def fp_res(x: str):
        x = parse.unquote(x)
        return x in glob.glob(os.path.join(watch_dir, pattern), recursive=True)

    return fp_res