Hook into GitLab's internal/post_receive notifications

Background

High frequency polling for updates may put unnecessary stress on gitaly.

As noted by @jacobvosmaer-gitlab in gitlab-com/gl-infra/readiness!46 (comment 426331665), gitaly already has a robust notification mechanism in place for gitlab-rails.

This is done via the internal/post_receive API endpoint.

Error budget impact

As of mid-March, we are rate-limiting ourselves and eating up the error budget. This should fix it.

Proposal

The bullet points below without a prefix are implement in KAS or Rails if rails is specified.

1. (done) Implement new gRPC server module in KAS (that is also exposed to Ruby) that provides a method to notify KAS on Git pushes (commits, branches, tags).
   1. gRPC method checks if event is relevant for any agent configurations
   2. gRPC method publishes event to all KAS replicas via Redis
2. (done) rails Implement new Kas::Client method that uses the aforementioned gRPC call notify KAS about git hooks
   1. the data to sent must be at least the full Git ref and the project id
3. (done) rails Implement new git hook service based on Git::BaseHookService which passes the Git hook event data (full ref and project id) to the aforementioned Kas::Client method
   1. we may or may not filter if the project actually configured any agent (we probably don't want to put load on DB, but it's only a read operation ...) KAS will filter.
4. (done) Implement service in KAS that allows for a generic pub / sub from Redis (maybe in modserver)
   1. Should be able to publish any proto.Message packaged in a anypb.Any to preserve type information
   2. Subscribers should register a callback
   3. Service unpacks received events from anypb.Any, validate and call all callbacks
5. (done) GitOps and Configuration modules subscribe to "commit push" broadcasts for each configuration/gitops polling loop they are running. The callback should determine if the event is relevant to this particular polling loop and publish the event into a channel in that case. Then the polling loop gets the event and polls immediately.
6. (done) Our polling function may need to be refactored to allow providing an extra channel for "poking" it to make another iteration. Ok going into too much detail here, going to stop.