Jekyll2024-04-10T19:04:20+00:00/feed.xmlmoscicky.devMisc stuff by Maciej Mościcki.Unlocking Kafka’s Potential: Tackling Tail Latency with eBPF2024-04-10T19:00:00+00:002024-04-10T19:00:00+00:00/performance/2024/04/10/unlocking-kafkas-potentialIn March, together with my colleague Piotr Rżysko we have published an article about Kafka performance analysis using eBPF: https://blog.allegro.tech/2024/03/kafka-performance-analysis.html.

I won’t be going into technical details here (I encourage you read the original piece) instead I would like to offer some random thoughts around it:

  • I think that Brendan Gregg’s System Performance is truly one of the most inspiring and valuable CS books I have read. I have first learned about eBPF from it and it provides a lot of performance tuning methodologies and does a solid job at explaining key OS concepts. For me it is one of 2 books from the last ~5 years that I would recommend every Software Engineer to read (the other one would be Martin Kleppmann’s Desining Data Intensive Applications).
  • I’ve seen a couple of comments saying that the whole analysis was pointless because Kafka docs are explicitly saying that XFS may be a better choice - we could have simply tried XFS first! I’d argue that such an approach is a variant of Streetlight Anti-Method described by Brendan Gregg (i.e. looking for the keys not were you lost them but where the light is best). Sure, trying random options known without measuring performance and analysing the root cause may work. But you can also end up spending days tweaking random permutations of different parameters hoping to find the right one.
  • Solving problems together is a great way to learn and make progress. This ma be an obvious one but it always fascinates me how effective working in a pair is.
]]>Building a Vector Search Engine2024-01-19T22:09:03+00:002024-01-19T22:09:03+00:00/meetups/2024/01/19/building-a-vector-search-engineIn January, I had a pleasure to present a talk titled “Building a Vector Search Engine” at the MLops meetup (MOPS) in Warsaw.

During the talk I covered the topics of:

  • approximate nearest neighbors search algorithms for faster search
  • vector compression techniques
  • indices evaluation

The presentation was not as fluent as I would like it to be and there were some mistakes here and there but practice makes perfect:)

You can watch the recording by clicking a thumbnail below:

Meetup recording

]]>