Continuous System Integration Testing

The Agncty Continuous System Integration Testing (CSIT) system design needs to meet the continuously expanding requirements of Agntcy projects including Agent Gateway Protocol, Agent Directory, and others.

Tests can be run locally using taskfile or in GitHub Actions.

The directory structure of the CSIT is the following:

csit
└── integrations
│   ├── Taskfile.yaml                   # Task definitions
│   ├── docs                            # Documentations
│   ├── environment
│   │   └── kind                        # kind related manifests
│   ├── agntcy-dir                      # Agent directory related tests, components, and so on
│   │   ├── components                  # The compontents charts
│   │   ├── examples                    # The examples that can be used for testing
│   │   ├── manifests                   # Requred manifests for the tests
│   │   └── tests                       # Tests
│   └── agntcy-agp                      # Agent Gateway related tests, components, and so on
│       └── agentic-apps                # Agentic apps for gateway tests
│           ├── autogen_agent
│           └── langchain_agent
│
└── samples
    ├── app1                            # Agentic application example
    │   ├── model.json                  # Required model file
    │   ├── build.config.yaml           # Required build configuration file
    ├── app2                            # Another agentic application example
    │   ├── model.json
    │   ├── build.config.yaml

Integration Tests

The integration tests are testing interactions between integrated components.

Directory structure

The CSIT integrations directory contains the tasks that create the test environment, deploy the components to be tested, and run the tests.

Running Integration Tests Locally

For running tests locally, we need to create a test cluster and deploy the test environment on it before running the tests. Make sure the following tools are installed:

To run tests locally:

Create the cluster and deploy the environment:

cd integrations
task kind:create
task test:env:directory:deploy

Run the tests:
```
task test:directory
```
When finished, the test cluster can be cleared:
```
task kind:destroy
```

Contributing Tests

Contributing your own tests to the project is a great way to improve the robustness and coverage of the testing suite.

To add your tests:

Fork and Clone the Repository

Fork the repository to your GitHub account. Clone your fork to your local machine.
```
git clone https://github.com/your-username/repository.git
cd repository
```
Create a new branch

Create a new branch for your additions to keep your changes organized and separate from the main codebase.
```
git checkout -b add-new-test
```
Navigate to the Integrations directory

Locate the integrations directory where the test components are organized.
```
cd integrations
```
Add your test

Following the existing structure, create a new sub-directory for your test if necessary. For example, integrations/new-component. Add all necessary test files, such as scripts, manifests, and configuration files.

Update Taskfile

Modify the Taskfile.yaml to include tasks for deploying and running your new test.

tasks:
  test:env:new-component:deploy:
    desc: Desription of deployig new component elements
    cmds:
      - # Command for deploying your components if needed

  test:env:new-component:cleanup:
    desc: Desription of cleaning up component elements
    cmds:
      - # Command for cleaning up your components if needed

  test:new-component:
    desc: Desription of the test
    cmds:
      - # Commands to set up and run your test

Test locally

Before pushing your changes, test them locally to ensure everything works as expected.

task kind:create
task test:env:new-componet:deploy
task test:new-component
task test:env:new-componet:cleanup
task kind:destroy

Document your test

Update the documentation in the docs folder to include details on the new test. Explain the purpose of the test, any special setup instructions, and how it fits into the overall testing strategy.
Commit and push your changes

Commit your changes with a descriptive message and push them to your fork.
```
git add .
git commit -m "feat: add new test for component X"
git push origin add-new-test
```
Submit a pull request

Go to the original repository on GitHub and submit a pull request from your branch. Provide a detailed description of what your test covers and any additional context needed for reviewers.

Samples

The samples directory in the CSIT repository serves two primary purposes related to the testing of agentic applications.

Running Samples Tests Locally

For running tests locally, we need the following tools to build the sample applications:

Run the test:

cd samples/[app-name]
task run:test

### Compilation and Execution Verification

The agentic applications stored within the `samples` directory are subjected to
sample tests. These tests are designed to run whenever changes are made to the
agentic apps to ensure they compile correctly and are able to execute as
expected.

### Base for Agent Directory Integration Test

The agentic applications in the `samples` directory also serve as the foundation
for the agent model build and push test. This specific test checks for the
presence of two required files: `model.json` and `build.config.yaml`. If these
files are present within an agentic application, the integration agent model
build and push tests are triggered. This test is crucial for validating the
construction and verification of the agent model, ensuring that all necessary
components are correctly configured and operational.