⚙️ Exercise 2.1: Ingest Knowledge Using the API

🎯 Exercise Overview

Go beyond manual upload! In this exercise, you'll use a ready-to-use API to automatically ingest Jira issues into your Dify knowledge base. This approach gives you structured documents with rich metadata—much more powerful than plain text files.

✨ This is the foundation for automated knowledge ingestion. Perfect for teams that want to integrate Jira data into their AI assistants!

🚀 What You'll Build

Connect to the deployed Jira ingestion API
Explore available Jira projects using Swagger UI
Ingest structured Jira issues into your Dify knowledge base
Compare manual vs. API-based ingestion results

📋 Step-by-Step Checklist

🎯 Exercise Checklist

Step 1: Get Dify API credentials
Step 2: Explore available projects
Step 3: Ingest Jira issues via API

Step 4: Verify ingestion in Dify
Step 5: Compare knowledge bases
Step 6: Test retrieval quality

⏱️ Estimated Time: 10-15 minutes | 🎯 Goal: API-ingested knowledge base with structured metadata

🚀 Why Use the API?

Manually uploading .txt files gives you a starting point, but API ingestion offers superpowers:

🧠 Rich documents with metadata (issue keys, project info, etc.)
🤖 Bulk upload multiple issues at once
✅ Structured data - each document linked to an issue_key
🧰 Automation ready - perfect for pipelines or daily ingestion

💡 Use this API if you want to:

Experiment with JSON ingestion instead of raw .txt
See how metadata improves search quality
Reuse this approach at work with real Jira projects

API Ingestion Example

🛠️ Step 1: Get Your Dify Configuration

📋 Configuration Steps

You'll need your Dify instance details to connect the API. Follow these steps:

1.1: Access Your Dify API Settings

In your Dify instance, navigate to Knowledge section
Find and copy your API Server URL
Click the API Key button to create a new API key for your datasets

Step 1: Access API settings in Knowledge section (click to expand)

Step 2: Create API key for datasets (click to expand)

🧠 Pro Tip: Keep your API Server URL and API key handy - you'll need them in the next steps!

🔍 Step 2: Explore Available Projects

📋 Exploration Steps

Before ingesting data, let's see what Jira projects are available using the interactive Swagger UI.

2.1: Open the API Documentation

📚 Interactive API Documentation

Explore available Jira projects and test API endpoints directly in your browser

🔗 Open Swagger UI →

The Swagger UI provides an interactive interface where you can:

📖 Browse all available endpoints
🧪 Test API calls directly in your browser
👀 See request/response examples
🔍 View available projects interactively

Swagger UI interface (click to expand)

💡 Tip: The Swagger UI is especially helpful if you're new to APIs. You can click "Try it out" on any endpoint to test it without writing curl commands!

📤 Step 3: Ingest Jira Issues

🔧 Ingestion Steps

Use the POST /jira/ingest endpoint to ingest specific projects. Leave the dataset_id empty and fill in your Dify configuration.

3.1: Using Command Line (curl)

Run this command to ingest a project:

curl -X 'POST' \
  'https://dify-jira.testingfantasy.com/jira/ingest?advanced_ingestion=false' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "request": {
    "project": "REST_JiraEcosystem_issues"
  },
  "dify_config": {
    "dify_base_url": "http://{your_dify_ip}/v1",
    "dify_api_key": "{your_dify_api_key}",
    "dataset_id": "" 
  }
}'

📝 Replace these values:

{your_dify_ip} → Your Dify instance URL (from Step 1)
{your_dify_api_key} → Your Dify API key (from Step 1)
REST_JiraEcosystem_issues → The project you want to ingest (use /projects endpoint to see all available projects)

3.2: Using Swagger UI (Recommended)

You can also use the Swagger UI to test the endpoint interactively:

Navigate to the POST /jira/ingest endpoint in Swagger UI
Click "Try it out"
Use this payload:

{
  "request": {
    "project": "REST_JiraEcosystem_issues"
  },
  "dify_config": {
    "dify_base_url": "http://{your_dify_ip}/v1",
    "dify_api_key": "{your_dify_api_key}",
    "dataset_id": "" 
  }
}

Click "Execute" to run the ingestion

Swagger UI ingest endpoint (click to expand)

✅ Step 4: Verify Ingestion in Dify

🔍 Verification Steps

After running the ingestion command, verify that your data was successfully uploaded to Dify.

4.1: Check the API Response

The API will return a response showing the ingestion status:

{
  "success": true,
  "dify_instance": "http://13.38.64.67/v1",
  "issues_ingested": 23,
  "project": "REST_JiraEcosystem_issues",
  "message": "Successfully ingested 23 issues from REST_JiraEcosystem_issues.json"
}

4.2: Verify in Dify Knowledge Base

Navigate to your Dify instance and check the Knowledge section. You should see your newly created knowledge base!

Knowledge base created via API (click to expand)

Dataset configuration (click to expand)

🎉 Success! Your Knowledge Base is Ready

Your Jira issues have been successfully ingested with structured metadata!

📚 Step 5: Compare Knowledge Base Approaches

🔬 Understanding the Differences

In this exercise, you will work with three knowledge bases in Dify:

📄 Manual Ingestion

Created by uploading .txt files manually. Simple but limited structure and no metadata.

🔧 Basic API Ingestion

API-based knowledge base with structured data. Useful for simple lookups and basic retrieval.

🚀 Advanced API Ingestion

Enhanced with query seeding and aliases. Better understanding of user intents and synonyms.

💡 Key Differences:

Manual: Raw text files, limited structure, no metadata
Basic API: Structured data with metadata (issue keys, project info)
Advanced API: Query seeding + aliases for better retrieval and understanding

🧪 Test These Queries on All Three Knowledge Bases:

👉 What is this issue about: Jira Issue: REST-259?
👉 Give a test plan for REST-266
👉 Who is the assignee for REST-266?
👉 Summarize the technical documentation for the REST module.

Notice how having better data and advanced techniques generates better results!

Comparing query results (click to expand)

Advanced query comparison (click to expand)

🧠 Pro Tip: After uploading, go to Retrieval testing in Dify and try the queries above. Compare the answers you get from each knowledge base. Structured data and advanced techniques enable smarter, more accurate answers!

🔍 What Happens Behind the Scenes?

🔬 Technical Details

When you ingest Jira issues via the API, here's what happens:

Each issue is converted into a descriptive document with summary, description, project, type, etc.
Smart chunking ensures readable chunks with overlap for better search coverage
Metadata (like issue_key) is attached to each chunk, improving traceability
Documents are vectorized and indexed in your Dify knowledge base

🔧 Advanced: Deploy Your Own API (Optional)

🚀 Deploy Your Own API Instance

If you want to deploy your own version of the API or understand how it works under the hood:

Visit https://github.com/bassagap/dify_jira and create a Codespace on the main branch
Follow the setup instructions to deploy your own instance
This gives you full control over the API and allows customization

Note: This is optional and only needed if you want to customize the API or understand the implementation details.

🎯 Exercise Complete! What's Next?

🎉 Congratulations!

You've successfully ingested Jira issues using the API!

✅ What You've Accomplished:

✅ Connected to the Jira ingestion API
✅ Explored available projects using Swagger UI
✅ Ingested structured Jira issues with metadata
✅ Compared different knowledge base approaches

🚀 Ready for the Next Challenge?

This deployed API is a great next step if you want to:

🤖 Automate ingestion in your organization
🔗 Connect to real Jira projects
🧪 Test richer search results with structured metadata
📊 Compare manual vs. API-based knowledge bases

🎯 Exercise Overview

🚀 What You'll Build

📋 Step-by-Step Checklist​

🎯 Exercise Checklist

🚀 Why Use the API?​

🛠️ Step 1: Get Your Dify Configuration​

📋 Configuration Steps

1.1: Access Your Dify API Settings​

🔍 Step 2: Explore Available Projects​