9 Commits

Author SHA1 Message Date
65fba21285 refactor(app): split the main command file into multiples
Some checks failed
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is failing
Signed-off-by: kjuulh <contact@kjuulh.io>
2023-08-08 19:41:19 +02:00
cuddle-please
d721b74d05 chore(release): 0.1.1
Some checks failed
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2023-08-08 14:22:50 +00:00
4c095d62e3 chore(cuddle-please): update
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
Signed-off-by: kjuulh <contact@kjuulh.io>
2023-08-08 16:20:29 +02:00
047f830b58 docs(README): add motivation why this project should exist
Some checks failed
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is failing
Signed-off-by: kjuulh <contact@kjuulh.io>
2023-08-08 16:09:54 +02:00
cuddle-please
80fe79d369 chore(release): 0.1.0
Some checks failed
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2023-08-08 14:02:12 +00:00
005433abee docs(README): added some more milestones and fixed docker-compose
All checks were successful
continuous-integration/drone/push Build is passing
Signed-off-by: kjuulh <contact@kjuulh.io>
2023-08-08 16:01:26 +02:00
9db86258fd docs: add clarification about expectations and what milestones are missing
All checks were successful
continuous-integration/drone/push Build is passing
Signed-off-by: kjuulh <contact@kjuulh.io>
2023-08-08 15:58:54 +02:00
617166ae00 Merge pull request 'Configure Renovate' (#1) from renovate/configure into main
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: https://git.front.kjuulh.io/kjuulh/contractor/pulls/1
2023-08-08 13:53:37 +00:00
6132978c1f Add renovate.json
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-08-08 13:51:12 +00:00
12 changed files with 861 additions and 679 deletions

37
CHANGELOG.md Normal file
View File

@@ -0,0 +1,37 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.1.1] - 2023-08-08
### Docs
- *(README)* add motivation why this project should exist
### Other
- *(cuddle-please)* update
## [0.1.0] - 2023-08-08
### Added
- add basic ci
- add basic readme
- with main loop
- with initial cmd and server
### Docs
- *(README)* added some more milestones and fixed docker-compose
- add clarification about expectations and what milestones are missing
- add license
- clarification
### Other
- Merge pull request 'Configure Renovate' (#1) from renovate/configure into main
Reviewed-on: https://git.front.kjuulh.io/kjuulh/contractor/pulls/1
- Add renovate.json

View File

@@ -1,21 +1,64 @@
# Contractor - A renovate bot for gitea and github
Contractor is a chatops like bot, integrating with github/gitea issues, allowing commands to trigger renovate runs.
Contractor is a chatops like bot, integrating with github/gitea issues, allowing
commands to trigger renovate runs.
```bash
/contractor refresh
```
Contractor runs in a regular docker image and uses the official renovate slim image behind the scenes, this can be changed in the configuration.
Contractor runs in a regular docker image and uses the official renovate slim
image behind the scenes, this can be changed in the configuration.
![command](./assets/command.png)
<small>Do note that the contractor was run under a personal user, hence the same user replied</small>
<small>Do note that the contractor was run under a personal user, hence the same
user replied</small>
## Motivation
Renovate by default if hosted yourself, is neither sharded, or runs on a
cron-job cycle. This leaves a lot to be desired from a developers point of view.
As it may take quite a long time for renovate to revisit the pull-request again,
if there is a lot of repositories enabled.
This project intends to add an ad-hoc invocation of renovate for a single
repository, this enables developers to retrigger renovate whenever they want.
The project is built to be integrated with github and gitea (initially), and
work in its pull-request system, so when a renovate pr shows up, you can either
manually retrigger it, or enable any of the options in the renovate dashboard,
and retrigger.
## DISCLAIMER
The project is still 0.x.x As such the api is subject to change, and the
examples will probably be out of date. The below should be seen as an example of
what the project will look like once feature-complete.
## Milestones
- [x] 0.1.0
- Includes basic setup such as working server bot, and installation command,
automation is missing however. Also only gitea support for now, because this
is where the project initially is supposed to be in use.
- [ ] 0.2.0
- Add GitHub support
- [ ] 0.3.0
- Add Delegation support (not clustering, just delegation of renovate jobs)
- [ ] 0.4.0
- Slack integration
- [ ] 0.5.0
- GitHub App and such support
- [ ] 0.6.0
- Add api key support
## Getting started
First you need to pull and run the contractor image, docker is the preferred way of execution, but golang is also available from source.
First you need to pull and run the contractor image, docker is the preferred way
of execution, but golang is also available from source.
Docker compose is given as an example, but you're free to run using `docker run` if you prefer.
Docker compose is given as an example, but you're free to run using `docker run`
if you prefer.
See example for a ready-to-run image
@@ -35,7 +78,6 @@ services:
- .env
```
```bash
# file: .env
GITEA_RENOVATE_TOKEN=<gitea application token> # needs repo and pull request permissions
@@ -57,15 +99,19 @@ CONTRACTOR_API_KEY='<some sufficiently secret password used for webhooks to auth
// Remember to fill out the options as you see fit, this is not a complete example
```
Use renovate secret for each `{{ secrets.HARBOR_SERVER_PASSWORD }}` in your config, replace `HARBOR_SERVER_PASSWORD` with your own
Use renovate secret for each `{{ secrets.HARBOR_SERVER_PASSWORD }}` in your
config, replace `HARBOR_SERVER_PASSWORD` with your own
And then run the server with: `docker compose up`
This has started the server, but github doesn't know that it needs to talk to you yet.
This has started the server, but github doesn't know that it needs to talk to
you yet.
As such host the server somewhere with a public hostname, such that github or gitea webhooks can reach it, i.e. contractor.some-domain.com:9111
As such host the server somewhere with a public hostname, such that github or
gitea webhooks can reach it, i.e. contractor.some-domain.com:9111
To install the webhook, either use the docker image, or download the cli from source.
To install the webhook, either use the docker image, or download the cli from
source.
### CLI
@@ -75,10 +121,15 @@ To install the cli
go install git.front.kjuulh.io/kjuulh/contractor@latest
```
contractor will automatically read any .env file, so you can leave out the secrets.
contractor will automatically read any .env file, so you can leave out the
secrets.
```bash
contractor install --owner kjuulh --repository contractor --url https://git.front.kjuulh.io/api/v1 --backend gitea
contractor install \
--owner kjuulh \
--repository contractor \
--url https://git.front.kjuulh.io/api/v1 \
--backend gitea
```
If you leave any of these out, contractor will prompt your for required values.
@@ -88,9 +139,15 @@ If you leave any of these out, contractor will prompt your for required values.
You can also use docker for it.
```bash
docker compose run contractor install --owner kjuulh --repository contractor --url https://git.front.kjuulh.io/api/v1 --backend gitea
docker compose run contractor \
install \
--owner kjuulh \
--repository contractor \
--url https://git.front.kjuulh.io/api/v1 \
--backend gitea
```
### GitHub App
TBD, this should automatically install the webhook for allowed repositories, I just haven't gotten around to it yet. It is on the 0.3.0 Roadmap.
TBD, this should automatically install the webhook for allowed repositories, I
just haven't gotten around to it yet. It is on the 0.3.0 Roadmap.

View File

@@ -195,7 +195,7 @@ mod please_release {
) -> eyre::Result<()> {
let build_image = client
.container()
.from("kasperhermansen/cuddle-please:main-1691463075");
.from("kasperhermansen/cuddle-please:main-1691504183");
let src = client
.git_opts(

View File

@@ -1,34 +1,17 @@
package contractor
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"html"
"io"
"log"
"net/http"
"os"
"strings"
"sync"
"time"
"dagger.io/dagger"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
"github.com/spf13/cobra"
)
type createHook struct {
Active bool `json:"active"`
AuthorizationHeader string `json:"authorization_header"`
BranchFilter string `json:"branch_filter"`
Config map[string]string `json:"config"`
Events []string `json:"events"`
Type string `json:"type"`
}
"git.front.kjuulh.io/kjuulh/contractor/internal/bot"
"git.front.kjuulh.io/kjuulh/contractor/internal/features"
"git.front.kjuulh.io/kjuulh/contractor/internal/providers"
"git.front.kjuulh.io/kjuulh/contractor/internal/queue"
"git.front.kjuulh.io/kjuulh/contractor/internal/renovate"
)
func installCmd() *cobra.Command {
var (
@@ -44,7 +27,7 @@ func installCmd() *cobra.Command {
Use: "install",
Run: func(cmd *cobra.Command, args []string) {
if err := NewGiteaClient(&url, &token).CreateWebhook(owner, repository); err != nil {
if err := providers.NewGiteaClient(&url, &token).CreateWebhook(owner, repository); err != nil {
log.Printf("failed to add create webhook: %s", err.Error())
}
},
@@ -69,77 +52,14 @@ func serverCmd() *cobra.Command {
token string
)
giteaClient := NewGiteaClient(&url, &token)
renovateClient := NewRenovateClient("")
queue := NewGoQueue()
queue.Subscribe(
MessageTypeRefreshRepository,
func(ctx context.Context, item *QueueMessage) error {
log.Printf("handling message: %s, content: %s", item.Type, item.Content)
return nil
},
)
queue.Subscribe(
MessageTypeRefreshRepositoryDone,
func(ctx context.Context, item *QueueMessage) error {
log.Printf("handling message: %s, content: %s", item.Type, item.Content)
return nil
},
)
queue.Subscribe(
MessageTypeRefreshRepository,
func(ctx context.Context, item *QueueMessage) error {
var request RefreshRepositoryRequest
if err := json.Unmarshal([]byte(item.Content), &request); err != nil {
log.Printf("failed to unmarshal request body: %s", err.Error())
return err
}
giteaClient := providers.NewGiteaClient(&url, &token)
renovateClient := renovate.NewRenovateClient("")
queue := queue.NewGoQueue()
botHandler := bot.NewBotHandler(giteaClient)
cancelCtx, cancel := context.WithTimeout(ctx, time.Second*30)
defer cancel()
giteaWebhook := features.NewGiteaWebhook(botHandler, queue)
if err := renovateClient.RefreshRepository(cancelCtx, request.Owner, request.Repository); err != nil {
queue.Insert(MessageTypeRefreshRepositoryDone, RefreshDoneRepositoryRequest{
Repository: request.Repository,
Owner: request.Owner,
PullRequestID: request.PullRequestID,
CommentID: request.CommentID,
CommentBody: request.CommentBody,
ReportProgress: request.ReportProgress,
Status: "failed",
Error: err.Error(),
})
return err
}
queue.Insert(MessageTypeRefreshRepositoryDone, RefreshDoneRepositoryRequest{
Repository: request.Repository,
Owner: request.Owner,
PullRequestID: request.PullRequestID,
CommentID: request.CommentID,
CommentBody: request.CommentBody,
ReportProgress: request.ReportProgress,
Status: "done",
Error: "",
})
return nil
},
)
queue.Subscribe(
MessageTypeRefreshRepositoryDone,
func(ctx context.Context, item *QueueMessage) error {
var doneRequest RefreshDoneRepositoryRequest
if err := json.Unmarshal([]byte(item.Content), &doneRequest); err != nil {
log.Printf("failed to unmarshal request body: %s", err.Error())
return err
}
return giteaClient.EditComment(ctx, &doneRequest)
},
)
features.RegisterGiteaQueues(queue, renovateClient, giteaClient)
cmd := &cobra.Command{
Use: "server",
@@ -148,41 +68,15 @@ func serverCmd() *cobra.Command {
cmd.PersistentFlags().StringVar(&url, "url", "", "the api url of the server")
cmd.PersistentFlags().StringVar(&token, "token", "", "the token to authenticate with")
cmd.AddCommand(serverServeCmd(&url, &token, queue, giteaClient))
cmd.AddCommand(serverServeCmd(&url, &token, giteaWebhook))
return cmd
}
const (
MessageTypeRefreshRepository = "refresh_repository"
MessageTypeRefreshRepositoryDone = "refresh_repository_done"
)
type RefreshRepositoryRequest struct {
Repository string `json:"repository"`
Owner string `json:"owner"`
PullRequestID int `json:"pullRequestId"`
CommentID int `json:"commentId"`
CommentBody string `json:"commentBody"`
ReportProgress bool `json:"reportProgress"`
}
type RefreshDoneRepositoryRequest struct {
Repository string `json:"repository"`
Owner string `json:"owner"`
PullRequestID int `json:"pullRequestId"`
CommentID int `json:"commentId"`
CommentBody string `json:"commentBody"`
ReportProgress bool `json:"reportProgress"`
Status string `json:"status"`
Error string `json:"error"`
}
func serverServeCmd(
url *string,
token *string,
queue *GoQueue,
giteaClient *GiteaClient,
giteaWebhook *features.GiteaWebhook,
) *cobra.Command {
cmd := &cobra.Command{
Use: "serve",
@@ -192,67 +86,19 @@ func serverServeCmd(
gitea := engine.Group("/gitea")
{
gitea.POST("/webhook", func(ctx *gin.Context) {
log.Println("received")
type GiteaWebhookRequest struct {
Action string `json:"action"`
Issue struct {
Id int `json:"id"`
Number int `json:"number"`
} `json:"issue"`
Comment struct {
Body string `json:"body"`
} `json:"comment"`
Repository struct {
FullName string `json:"full_name"`
}
}
var request GiteaWebhookRequest
var request features.GiteaWebhookRequest
if err := ctx.BindJSON(&request); err != nil {
ctx.AbortWithError(500, err)
return
}
command, ok := validateBotComment(request.Comment.Body)
if ok {
log.Printf("got webhook request: contractor %s", command)
bot := NewBotHandler(giteaClient)
output, err := bot.Handle(command)
if err != nil {
log.Printf("failed to run bot handler with error: %s", err.Error())
}
parts := strings.Split(request.Repository.FullName, "/")
comment, err := bot.AppendComment(
parts[0],
parts[1],
request.Issue.Number,
output,
)
if err != nil {
ctx.AbortWithError(500, err)
return
}
if err := queue.Insert(MessageTypeRefreshRepository, RefreshRepositoryRequest{
Repository: parts[1],
Owner: parts[0],
PullRequestID: request.Issue.Number,
CommentID: comment.ID,
CommentBody: comment.Body,
ReportProgress: true,
}); err != nil {
if err := giteaWebhook.HandleGiteaWebhook(ctx.Request.Context(), &request); err != nil {
ctx.AbortWithError(500, err)
return
}
ctx.Status(204)
}
})
}
@@ -263,14 +109,6 @@ func serverServeCmd(
return cmd
}
func validateBotComment(s string) (request string, ok bool) {
if after, ok := strings.CutPrefix(s, "/contractor"); ok {
return strings.TrimSpace(after), true
}
return "", false
}
func RootCmd() *cobra.Command {
cmd := &cobra.Command{Use: "contractor"}
@@ -278,473 +116,3 @@ func RootCmd() *cobra.Command {
return cmd
}
type BotHandler struct {
giteaClient *GiteaClient
}
func NewBotHandler(gitea *GiteaClient) *BotHandler {
return &BotHandler{giteaClient: gitea}
}
func (b *BotHandler) Handle(input string) (output string, err error) {
innerHandle := func(input string) (output string, err error) {
if strings.HasPrefix(input, "help") {
return b.Help(), nil
}
if strings.HasPrefix(input, "refresh") {
return `
<h3>Contractor triggered renovate refresh on this repository</h3>
This comment will be updated with status
<!-- Status update start -->
<!-- Status update end -->
`, nil
}
return b.Help(), errors.New("could not recognize command")
}
output, err = innerHandle(input)
output = fmt.Sprintf(
"%s\n<small>This comment was generated by <a href='https://git.front.kjuulh.io/kjuulh/contractor'>Contractor</a></small>",
output,
)
return output, err
}
func (b *BotHandler) Help() string {
return `<details open>
<summary><h3>/contractor [command]</h3></summary>
<strong>Commands:</strong>
* /contractor help
* triggers the help menu
* /contractor refresh
* triggers renovate to refresh the current pull request
</details>`
}
type AddCommentResponse struct {
Body string `json:"body"`
ID int `json:"id"`
}
func (b *BotHandler) AppendComment(
owner string,
repository string,
pullRequest int,
comment string,
) (*AddCommentResponse, error) {
return b.giteaClient.AddComment(owner, repository, pullRequest, comment)
}
type QueueMessage struct {
Type string `json:"type"`
Content string `json:"content"`
}
type GoQueue struct {
queue []*QueueMessage
queueLock sync.Mutex
subscribers map[string]map[string]func(ctx context.Context, item *QueueMessage) error
subscribersLock sync.RWMutex
}
func NewGoQueue() *GoQueue {
return &GoQueue{
queue: make([]*QueueMessage, 0),
subscribers: make(
map[string]map[string]func(ctx context.Context, item *QueueMessage) error,
),
}
}
func (gq *GoQueue) Subscribe(
messageType string,
callback func(ctx context.Context, item *QueueMessage) error,
) string {
gq.subscribersLock.Lock()
defer gq.subscribersLock.Unlock()
uid, err := uuid.NewUUID()
if err != nil {
panic(err)
}
id := uid.String()
_, ok := gq.subscribers[messageType]
if !ok {
messageTypeSubscribers := make(
map[string]func(ctx context.Context, item *QueueMessage) error,
)
messageTypeSubscribers[id] = callback
gq.subscribers[messageType] = messageTypeSubscribers
} else {
gq.subscribers[messageType][id] = callback
}
return id
}
func (gq *GoQueue) Unsubscribe(messageType string, id string) {
gq.subscribersLock.Lock()
defer gq.subscribersLock.Unlock()
_, ok := gq.subscribers[messageType]
if !ok {
// No work to be done
return
} else {
delete(gq.subscribers[messageType], id)
}
}
func (gq *GoQueue) Insert(messageType string, content any) error {
gq.queueLock.Lock()
defer gq.queueLock.Unlock()
contents, err := json.Marshal(content)
if err != nil {
return err
}
gq.queue = append(gq.queue, &QueueMessage{
Type: messageType,
Content: string(contents),
})
go func() {
gq.handle(context.Background())
}()
return nil
}
func (gq *GoQueue) handle(ctx context.Context) {
gq.queueLock.Lock()
defer gq.queueLock.Unlock()
for {
if len(gq.queue) == 0 {
return
}
item := gq.queue[0]
gq.queue = gq.queue[1:]
gq.subscribersLock.RLock()
defer gq.subscribersLock.RUnlock()
for id, callback := range gq.subscribers[item.Type] {
log.Printf("sending message to %s", id)
go callback(ctx, item)
}
}
}
type GiteaClient struct {
url *string
token *string
client *http.Client
}
func NewGiteaClient(url, token *string) *GiteaClient {
return &GiteaClient{
url: url,
token: token,
client: http.DefaultClient,
}
}
func (gc *GiteaClient) EditComment(
ctx context.Context,
doneRequest *RefreshDoneRepositoryRequest,
) error {
commentBody := html.UnescapeString(doneRequest.CommentBody)
startCmnt := "<!-- Status update start -->"
startIdx := strings.Index(commentBody, startCmnt)
endIdx := strings.Index(commentBody, "<!-- Status update end -->")
if startIdx >= 0 && endIdx >= 0 {
log.Println("found comment to replace")
var content string
if doneRequest.Error != "" {
content = fmt.Sprintf("<pre>ERROR: %s</pre><br>", doneRequest.Error)
}
if doneRequest.Status != "" {
content = fmt.Sprintf("<p>%s</p>", doneRequest.Status)
}
doneRequest.CommentBody = fmt.Sprintf(
"%s<br><hr>%s<hr><br>%s",
commentBody[:startIdx+len(startCmnt)],
content,
commentBody[endIdx:],
)
}
editComment := struct {
Body string `json:"body"`
}{
Body: doneRequest.CommentBody,
}
body, err := json.Marshal(editComment)
if err != nil {
log.Println("failed to marshal request body: %w", err)
return err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPatch,
fmt.Sprintf(
"%s/repos/%s/%s/issues/comments/%d",
strings.TrimSuffix(*gc.url, "/"),
doneRequest.Owner,
doneRequest.Repository,
doneRequest.CommentID,
),
bodyReader,
)
if err != nil {
log.Printf("failed to form update comment request: %s", err.Error())
return err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
log.Printf("failed to update comment: %s", err.Error())
return err
}
if resp.StatusCode > 299 {
log.Printf("failed to update comment with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
log.Printf("failed to read body of error response: %s", err.Error())
} else {
log.Printf("request body: %s", string(respBody))
}
}
return nil
}
func (gc *GiteaClient) CreateWebhook(owner, repository string) error {
createHookOptions := createHook{
Active: true,
AuthorizationHeader: "",
BranchFilter: "*",
Config: map[string]string{
"url": "http://10.0.9.1:8080/gitea/webhook",
"content_type": "json",
},
Events: []string{
"pull_request_comment",
},
Type: "gitea",
}
body, err := json.Marshal(createHookOptions)
if err != nil {
log.Println("failed to marshal request body: %w", err)
return err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPost,
fmt.Sprintf(
"%s/repos/%s/%s/hooks",
strings.TrimSuffix(*gc.url, "/"),
owner,
repository,
),
bodyReader,
)
if err != nil {
log.Printf("failed to form create hook request: %s", err.Error())
return err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
log.Printf("failed to register hook: %s", err.Error())
return err
}
if resp.StatusCode > 299 {
log.Printf("failed to register with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
log.Printf("failed to read body of error response: %s", err.Error())
} else {
log.Printf("request body: %s", string(respBody))
}
}
return nil
}
func (gc *GiteaClient) AddComment(
owner, repository string,
pullRequest int,
comment string,
) (*AddCommentResponse, error) {
addComment := struct {
Body string `json:"body"`
}{
Body: comment,
}
body, err := json.Marshal(addComment)
if err != nil {
return nil, err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPost,
fmt.Sprintf(
"%s/repos/%s/%s/issues/%d/comments",
strings.TrimSuffix(*gc.url, "/"),
owner,
repository,
pullRequest,
),
bodyReader,
)
if err != nil {
return nil, err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
return nil, err
}
if resp.StatusCode > 299 {
log.Printf("failed to register with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
} else {
log.Printf("request body: %s", string(respBody))
}
}
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var response AddCommentResponse
if err := json.Unmarshal(respBody, &response); err != nil {
return nil, err
}
return &response, nil
}
type RenovateClient struct {
config string
}
func NewRenovateClient(config string) *RenovateClient {
return &RenovateClient{config: config}
}
func (rc *RenovateClient) RefreshRepository(ctx context.Context, owner, repository string) error {
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
return err
}
envRenovateToken := os.Getenv("GITEA_RENOVATE_TOKEN")
log.Println(envRenovateToken)
renovateToken := client.SetSecret("RENOVATE_TOKEN", envRenovateToken)
githubComToken := client.SetSecret("GITHUB_COM_TOKEN", os.Getenv("GITHUB_COM_TOKEN"))
renovateSecret := client.SetSecret("RENOVATE_SECRETS", os.Getenv("RENOVATE_SECRETS"))
output, err := client.Container().
From("renovate/renovate:latest").
WithNewFile("/opts/renovate/config.json", dagger.ContainerWithNewFileOpts{
Contents: `{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"platform": "gitea",
"endpoint": "https://git.front.kjuulh.io/api/v1/",
"automerge": true,
"automergeType": "pr",
"extends": [
"config:base"
],
"hostRules": [
{
"hostType": "docker",
"matchHost": "harbor.front.kjuulh.io",
"username": "service",
"password": "{{ secrets.HARBOR_SERVER_PASSWORD }}"
}
],
"packageRules": [
{
"matchDatasources": ["docker"],
"registryUrls": ["https://harbor.front.kjuulh.io/docker-proxy/library/"]
},
{
"groupName": "all dependencies",
"separateMajorMinor": false,
"groupSlug": "all",
"packageRules": [
{
"matchPackagePatterns": [
"*"
],
"groupName": "all dependencies",
"groupSlug": "all"
}
],
"lockFileMaintenance": {
"enabled": false
}
}
]
}`,
Permissions: 755,
Owner: "root",
}).
WithSecretVariable("RENOVATE_TOKEN", renovateToken).
WithSecretVariable("GITHUB_COM_TOKEN", githubComToken).
WithSecretVariable("RENOVATE_SECRETS", renovateSecret).
WithEnvVariable("LOG_LEVEL", "warn").
WithEnvVariable("RENOVATE_CONFIG_FILE", "/opts/renovate/config.json").
WithExec([]string{
fmt.Sprintf("%s/%s", owner, repository),
}).
Sync(ctx)
stdout, outerr := output.Stdout(ctx)
if outerr == nil {
log.Printf("stdout: %s", stdout)
}
stderr, outerr := output.Stderr(ctx)
if outerr == nil {
log.Printf("stderr: %s", stderr)
}
if err != nil {
return fmt.Errorf("error: %w, \nstderr: %s\nstdout: %s", err, stderr, stdout)
}
return nil
}

67
internal/bot/giteabot.go Normal file
View File

@@ -0,0 +1,67 @@
package bot
import (
"errors"
"fmt"
"strings"
"git.front.kjuulh.io/kjuulh/contractor/internal/models"
"git.front.kjuulh.io/kjuulh/contractor/internal/providers"
)
type BotHandler struct {
giteaClient *providers.GiteaClient
}
func NewBotHandler(gitea *providers.GiteaClient) *BotHandler {
return &BotHandler{giteaClient: gitea}
}
func (b *BotHandler) Handle(input string) (output string, err error) {
innerHandle := func(input string) (output string, err error) {
if strings.HasPrefix(input, "help") {
return b.Help(), nil
}
if strings.HasPrefix(input, "refresh") {
return `
<h3>Contractor triggered renovate refresh on this repository</h3>
This comment will be updated with status
<!-- Status update start -->
<!-- Status update end -->
`, nil
}
return b.Help(), errors.New("could not recognize command")
}
output, err = innerHandle(input)
output = fmt.Sprintf(
"%s\n<small>This comment was generated by <a href='https://git.front.kjuulh.io/kjuulh/contractor'>Contractor</a></small>",
output,
)
return output, err
}
func (b *BotHandler) Help() string {
return `<details open>
<summary><h3>/contractor [command]</h3></summary>
<strong>Commands:</strong>
* /contractor help
* triggers the help menu
* /contractor refresh
* triggers renovate to refresh the current pull request
</details>`
}
func (b *BotHandler) AppendComment(
owner string,
repository string,
pullRequest int,
comment string,
) (*models.AddCommentResponse, error) {
return b.giteaClient.AddComment(owner, repository, pullRequest, comment)
}

View File

@@ -0,0 +1,84 @@
package features
import (
"context"
"encoding/json"
"log"
"time"
"git.front.kjuulh.io/kjuulh/contractor/internal/models"
"git.front.kjuulh.io/kjuulh/contractor/internal/providers"
"git.front.kjuulh.io/kjuulh/contractor/internal/queue"
"git.front.kjuulh.io/kjuulh/contractor/internal/renovate"
)
func RegisterGiteaQueues(goqueue *queue.GoQueue, renovate *renovate.RenovateClient, giteaClient *providers.GiteaClient) {
goqueue.Subscribe(
models.MessageTypeRefreshRepository,
func(ctx context.Context, item *queue.QueueMessage) error {
log.Printf("handling message: %s, content: %s", item.Type, item.Content)
return nil
},
)
goqueue.Subscribe(
models.MessageTypeRefreshRepositoryDone,
func(ctx context.Context, item *queue.QueueMessage) error {
log.Printf("handling message: %s, content: %s", item.Type, item.Content)
return nil
},
)
goqueue.Subscribe(
models.MessageTypeRefreshRepository,
func(ctx context.Context, item *queue.QueueMessage) error {
var request models.RefreshRepositoryRequest
if err := json.Unmarshal([]byte(item.Content), &request); err != nil {
log.Printf("failed to unmarshal request body: %s", err.Error())
return err
}
cancelCtx, cancel := context.WithTimeout(ctx, time.Minute*5)
defer cancel()
if err := renovate.RefreshRepository(cancelCtx, request.Owner, request.Repository); err != nil {
goqueue.Insert(models.MessageTypeRefreshRepositoryDone, models.RefreshDoneRepositoryRequest{
Repository: request.Repository,
Owner: request.Owner,
PullRequestID: request.PullRequestID,
CommentID: request.CommentID,
CommentBody: request.CommentBody,
ReportProgress: request.ReportProgress,
Status: "failed",
Error: err.Error(),
})
return err
}
goqueue.Insert(models.MessageTypeRefreshRepositoryDone, models.RefreshDoneRepositoryRequest{
Repository: request.Repository,
Owner: request.Owner,
PullRequestID: request.PullRequestID,
CommentID: request.CommentID,
CommentBody: request.CommentBody,
ReportProgress: request.ReportProgress,
Status: "done",
Error: "",
})
return nil
},
)
goqueue.Subscribe(
models.MessageTypeRefreshRepositoryDone,
func(ctx context.Context, item *queue.QueueMessage) error {
var doneRequest models.RefreshDoneRepositoryRequest
if err := json.Unmarshal([]byte(item.Content), &doneRequest); err != nil {
log.Printf("failed to unmarshal request body: %s", err.Error())
return err
}
return giteaClient.EditComment(ctx, &doneRequest)
},
)
}

View File

@@ -0,0 +1,83 @@
package features
import (
"context"
"log"
"strings"
"git.front.kjuulh.io/kjuulh/contractor/internal/bot"
"git.front.kjuulh.io/kjuulh/contractor/internal/models"
"git.front.kjuulh.io/kjuulh/contractor/internal/queue"
)
type GiteaWebhook struct {
botHandler *bot.BotHandler
queue *queue.GoQueue
}
type GiteaWebhookRequest struct {
Action string `json:"action"`
Issue struct {
Id int `json:"id"`
Number int `json:"number"`
} `json:"issue"`
Comment struct {
Body string `json:"body"`
} `json:"comment"`
Repository struct {
FullName string `json:"full_name"`
}
}
func NewGiteaWebhook(botHandler *bot.BotHandler, queue *queue.GoQueue) *GiteaWebhook {
return &GiteaWebhook{
botHandler: botHandler,
queue: queue,
}
}
func (gw *GiteaWebhook) HandleGiteaWebhook(ctx context.Context, request *GiteaWebhookRequest) error {
command, ok := validateBotComment(request.Comment.Body)
if ok {
log.Printf("got webhook request: contractor %s", command)
bot := gw.botHandler
output, err := bot.Handle(command)
if err != nil {
log.Printf("failed to run bot handler with error: %s", err.Error())
}
parts := strings.Split(request.Repository.FullName, "/")
comment, err := bot.AppendComment(
parts[0],
parts[1],
request.Issue.Number,
output,
)
if err != nil {
return err
}
if err := gw.queue.Insert(models.MessageTypeRefreshRepository, models.RefreshRepositoryRequest{
Repository: parts[1],
Owner: parts[0],
PullRequestID: request.Issue.Number,
CommentID: comment.ID,
CommentBody: comment.Body,
ReportProgress: true,
}); err != nil {
return err
}
}
return nil
}
func validateBotComment(s string) (request string, ok bool) {
if after, ok := strings.CutPrefix(s, "/contractor"); ok {
return strings.TrimSpace(after), true
}
return "", false
}

View File

@@ -0,0 +1,40 @@
package models
const (
MessageTypeRefreshRepository = "refresh_repository"
MessageTypeRefreshRepositoryDone = "refresh_repository_done"
)
type CreateHook struct {
Active bool `json:"active"`
AuthorizationHeader string `json:"authorization_header"`
BranchFilter string `json:"branch_filter"`
Config map[string]string `json:"config"`
Events []string `json:"events"`
Type string `json:"type"`
}
type RefreshRepositoryRequest struct {
Repository string `json:"repository"`
Owner string `json:"owner"`
PullRequestID int `json:"pullRequestId"`
CommentID int `json:"commentId"`
CommentBody string `json:"commentBody"`
ReportProgress bool `json:"reportProgress"`
}
type RefreshDoneRepositoryRequest struct {
Repository string `json:"repository"`
Owner string `json:"owner"`
PullRequestID int `json:"pullRequestId"`
CommentID int `json:"commentId"`
CommentBody string `json:"commentBody"`
ReportProgress bool `json:"reportProgress"`
Status string `json:"status"`
Error string `json:"error"`
}
type AddCommentResponse struct {
Body string `json:"body"`
ID int `json:"id"`
}

227
internal/providers/gitea.go Normal file
View File

@@ -0,0 +1,227 @@
package providers
import (
"bytes"
"context"
"encoding/json"
"fmt"
"html"
"io"
"log"
"net/http"
"strings"
"git.front.kjuulh.io/kjuulh/contractor/internal/models"
)
type GiteaClient struct {
url *string
token *string
client *http.Client
}
func NewGiteaClient(url, token *string) *GiteaClient {
return &GiteaClient{
url: url,
token: token,
client: http.DefaultClient,
}
}
func (gc *GiteaClient) EditComment(
ctx context.Context,
doneRequest *models.RefreshDoneRepositoryRequest,
) error {
commentBody := html.UnescapeString(doneRequest.CommentBody)
startCmnt := "<!-- Status update start -->"
startIdx := strings.Index(commentBody, startCmnt)
endIdx := strings.Index(commentBody, "<!-- Status update end -->")
if startIdx >= 0 && endIdx >= 0 {
log.Println("found comment to replace")
var content string
if doneRequest.Error != "" {
content = fmt.Sprintf("<pre>ERROR: %s</pre><br>", doneRequest.Error)
}
if doneRequest.Status != "" {
content = fmt.Sprintf("%s<p>%s</p>", content, doneRequest.Status)
}
doneRequest.CommentBody = fmt.Sprintf(
"%s<br><hr>%s<hr><br>%s",
commentBody[:startIdx+len(startCmnt)],
content,
commentBody[endIdx:],
)
}
editComment := struct {
Body string `json:"body"`
}{
Body: doneRequest.CommentBody,
}
body, err := json.Marshal(editComment)
if err != nil {
log.Println("failed to marshal request body: %w", err)
return err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPatch,
fmt.Sprintf(
"%s/repos/%s/%s/issues/comments/%d",
strings.TrimSuffix(*gc.url, "/"),
doneRequest.Owner,
doneRequest.Repository,
doneRequest.CommentID,
),
bodyReader,
)
if err != nil {
log.Printf("failed to form update comment request: %s", err.Error())
return err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
log.Printf("failed to update comment: %s", err.Error())
return err
}
if resp.StatusCode > 299 {
log.Printf("failed to update comment with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
log.Printf("failed to read body of error response: %s", err.Error())
} else {
log.Printf("request body: %s", string(respBody))
}
}
return nil
}
func (gc *GiteaClient) CreateWebhook(owner, repository string) error {
createHookOptions := models.CreateHook{
Active: true,
AuthorizationHeader: "",
BranchFilter: "*",
Config: map[string]string{
"url": "http://10.0.9.1:8080/gitea/webhook",
"content_type": "json",
},
Events: []string{
"pull_request_comment",
},
Type: "gitea",
}
body, err := json.Marshal(createHookOptions)
if err != nil {
log.Println("failed to marshal request body: %w", err)
return err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPost,
fmt.Sprintf(
"%s/repos/%s/%s/hooks",
strings.TrimSuffix(*gc.url, "/"),
owner,
repository,
),
bodyReader,
)
if err != nil {
log.Printf("failed to form create hook request: %s", err.Error())
return err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
log.Printf("failed to register hook: %s", err.Error())
return err
}
if resp.StatusCode > 299 {
log.Printf("failed to register with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
log.Printf("failed to read body of error response: %s", err.Error())
} else {
log.Printf("request body: %s", string(respBody))
}
}
return nil
}
func (gc *GiteaClient) AddComment(
owner, repository string,
pullRequest int,
comment string,
) (*models.AddCommentResponse, error) {
addComment := struct {
Body string `json:"body"`
}{
Body: comment,
}
body, err := json.Marshal(addComment)
if err != nil {
return nil, err
}
bodyReader := bytes.NewReader(body)
request, err := http.NewRequest(
http.MethodPost,
fmt.Sprintf(
"%s/repos/%s/%s/issues/%d/comments",
strings.TrimSuffix(*gc.url, "/"),
owner,
repository,
pullRequest,
),
bodyReader,
)
if err != nil {
return nil, err
}
request.Header.Add("Authorization", fmt.Sprintf("token %s", *gc.token))
request.Header.Add("Content-Type", "application/json")
resp, err := gc.client.Do(request)
if err != nil {
return nil, err
}
if resp.StatusCode > 299 {
log.Printf("failed to register with status code: %d", resp.StatusCode)
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
} else {
log.Printf("request body: %s", string(respBody))
}
}
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var response models.AddCommentResponse
if err := json.Unmarshal(respBody, &response); err != nil {
return nil, err
}
return &response, nil
}

113
internal/queue/goqueue.go Normal file
View File

@@ -0,0 +1,113 @@
package queue
import (
"context"
"encoding/json"
"log"
"sync"
"github.com/google/uuid"
)
type QueueMessage struct {
Type string `json:"type"`
Content string `json:"content"`
}
type GoQueue struct {
queue []*QueueMessage
queueLock sync.Mutex
subscribers map[string]map[string]func(ctx context.Context, item *QueueMessage) error
subscribersLock sync.RWMutex
}
func NewGoQueue() *GoQueue {
return &GoQueue{
queue: make([]*QueueMessage, 0),
subscribers: make(
map[string]map[string]func(ctx context.Context, item *QueueMessage) error,
),
}
}
func (gq *GoQueue) Subscribe(
messageType string,
callback func(ctx context.Context, item *QueueMessage) error,
) string {
gq.subscribersLock.Lock()
defer gq.subscribersLock.Unlock()
uid, err := uuid.NewUUID()
if err != nil {
panic(err)
}
id := uid.String()
_, ok := gq.subscribers[messageType]
if !ok {
messageTypeSubscribers := make(
map[string]func(ctx context.Context, item *QueueMessage) error,
)
messageTypeSubscribers[id] = callback
gq.subscribers[messageType] = messageTypeSubscribers
} else {
gq.subscribers[messageType][id] = callback
}
return id
}
func (gq *GoQueue) Unsubscribe(messageType string, id string) {
gq.subscribersLock.Lock()
defer gq.subscribersLock.Unlock()
_, ok := gq.subscribers[messageType]
if !ok {
// No work to be done
return
} else {
delete(gq.subscribers[messageType], id)
}
}
func (gq *GoQueue) Insert(messageType string, content any) error {
gq.queueLock.Lock()
defer gq.queueLock.Unlock()
contents, err := json.Marshal(content)
if err != nil {
return err
}
gq.queue = append(gq.queue, &QueueMessage{
Type: messageType,
Content: string(contents),
})
go func() {
gq.handle(context.Background())
}()
return nil
}
func (gq *GoQueue) handle(ctx context.Context) {
gq.queueLock.Lock()
defer gq.queueLock.Unlock()
for {
if len(gq.queue) == 0 {
return
}
item := gq.queue[0]
gq.queue = gq.queue[1:]
gq.subscribersLock.RLock()
defer gq.subscribersLock.RUnlock()
for id, callback := range gq.subscribers[item.Type] {
log.Printf("sending message to %s", id)
go callback(ctx, item)
}
}
}

103
internal/renovate/client.go Normal file
View File

@@ -0,0 +1,103 @@
package renovate
import (
"context"
"fmt"
"log"
"os"
"dagger.io/dagger"
)
type RenovateClient struct {
config string
}
func NewRenovateClient(config string) *RenovateClient {
return &RenovateClient{config: config}
}
func (rc *RenovateClient) RefreshRepository(ctx context.Context, owner, repository string) error {
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
return err
}
envRenovateToken := os.Getenv("GITEA_RENOVATE_TOKEN")
log.Println(envRenovateToken)
renovateToken := client.SetSecret("RENOVATE_TOKEN", envRenovateToken)
githubComToken := client.SetSecret("GITHUB_COM_TOKEN", os.Getenv("GITHUB_COM_TOKEN"))
renovateSecret := client.SetSecret("RENOVATE_SECRETS", os.Getenv("RENOVATE_SECRETS"))
output, err := client.Container().
From("renovate/renovate:latest").
WithNewFile("/opts/renovate/config.json", dagger.ContainerWithNewFileOpts{
Contents: `{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"platform": "gitea",
"endpoint": "https://git.front.kjuulh.io/api/v1/",
"automerge": true,
"automergeType": "pr",
"extends": [
"config:base"
],
"hostRules": [
{
"hostType": "docker",
"matchHost": "harbor.front.kjuulh.io",
"username": "service",
"password": "{{ secrets.HARBOR_SERVER_PASSWORD }}"
}
],
"packageRules": [
{
"matchDatasources": ["docker"],
"registryUrls": ["https://harbor.front.kjuulh.io/docker-proxy/library/"]
},
{
"groupName": "all dependencies",
"separateMajorMinor": false,
"groupSlug": "all",
"packageRules": [
{
"matchPackagePatterns": [
"*"
],
"groupName": "all dependencies",
"groupSlug": "all"
}
],
"lockFileMaintenance": {
"enabled": false
}
}
]
}`,
Permissions: 755,
Owner: "root",
}).
WithSecretVariable("RENOVATE_TOKEN", renovateToken).
WithSecretVariable("GITHUB_COM_TOKEN", githubComToken).
WithSecretVariable("RENOVATE_SECRETS", renovateSecret).
WithEnvVariable("LOG_LEVEL", "warn").
WithEnvVariable("RENOVATE_CONFIG_FILE", "/opts/renovate/config.json").
WithExec([]string{
fmt.Sprintf("%s/%s", owner, repository),
}).
Sync(ctx)
stdout, outerr := output.Stdout(ctx)
if outerr == nil {
log.Printf("stdout: %s", stdout)
}
stderr, outerr := output.Stderr(ctx)
if outerr == nil {
log.Printf("stderr: %s", stderr)
}
if err != nil {
return fmt.Errorf("error: %w, \nstderr: %s\nstdout: %s", err, stderr, stdout)
}
return nil
}

3
renovate.json Normal file
View File

@@ -0,0 +1,3 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json"
}