Subscribe to Personal Log via RSS and find other blogs to read in my blogroll.

A patch for a 14 years old project

So today I received a heart-warming email that ticks all the boxes for me:

  • First of all, they thank me for publishing the software on the Internet under the GPL license.
  • The author is user of one of my projects.
  • They are happy, bogom and the bogofilter are useful for us, I used it more than 10 years (sic).
  • And they are sending me a patch!

Bogom is a milter, which is a type of plugin for some MTAs –originally Sendmail, but later Postfix at least added support–, that is used to filter email.

My milter was an way of using bogofilter, a bayesian filter –yes, all this is about filtering– that can be used to filter spam. When I wrote the milter there were a few ways of using bogofilter with Sendmail, but all of them were filtering email after it was delivered. Bogom was able to filter and give a response to the MTA trying to deliver the email, so the sender knew that the email was rejected. I thought it was neat.

You didn’t have to use it to reject spam, because it could just add a header with the result of the classification, but that was the main idea. It had a few limitations, like only supporting a database for the whole system, but looks like by being simple, it has been functional for 14 years after I stopped working on it.

In fact, I have checked, and it still exist in the FreeBSD ports under mail/milter-bogom; and it doesn’t include any patches, so it is being used as I released it last time in 2008. Amazing!

The patch adds a flag to select the logging facility. Bogom uses daemon, and the idea is that you can use syslog to redirect the logs to any file you want, so the patch is not really that useful; but on the other hand, why not?

The only problem is that making a new release of a 14 years old unmaintained project would require testing of something that, honestly, I haven’t used for quite some time –not 14 years, but almost–. So we will see.

For now, this email has made my day!

An E/N blog

So I was having a chat with my friend Oscar when he mentioned an old post on his blog linking to one of my posts in my old Spanish blog, that doesn’t exist any more. That led us to discuss broken links, and how things have changed in the blogosphere.

I took down my old blog last year after 18 years online, because I thought it was irrelevant and its old archive was kind of holding me back to write other things. So long story short, I started this blog and that’s where we are now. Besides there were some technical reasons for the change, because the old blog was running on an old and somewhat unmaintained stack –all my fault: a hand crafted blogging engine built on top of Tornado, with Python 2, and using Redis as storage; don’t ask me why!–.

Anyway, Oscar has also moved from his hand crafted blog engine (with Django, if I’m not mistaken), to an static site generator. But he has decided to port his old posts to the new blog, despite being a considerable amount of work.

It is an interesting problem, but we both concluded that, in reality, nobody cares about what we write or not in our blogs. And perhaps it is a thought for a different post, considering if anybody ever cared about our blogs –even if we had a different perception back in 2003 when we started writing–.

That conversation reminded me of this quote from E/N:

The website’s content means everything to the publisher, but it could mean nothing to the rest of the world.

Which basically sums up what is this blog, and perhaps any personal blog.

I write about things that I find interesting, useful, or are important to me. That might be useful to anyone visiting the blog, but is not essential!

Changing hosts

Recently I upgraded my VPS to Debian Buster, that is not the latest version, but it’s OK as long as is long time support and it still has support time. It doesn’t take long –around 30 minutes–, but there are some tricky bits to it because between releases the software I use will change and I may need to adjust my configuration files –some of them come from my first install around 2010!–. That’s the main reason why I don’t do these upgrades ASAP.

I thought everything was good, until I noticed some weird behaviour, and turns out I had missed some kernel oops after the upgrade of one of them: iptables wasn’t working. That hosting provider injects a kernel they control, and it looks like there was a mismatch between it and my new Debian system. So I opened a support ticket, because the kernel is from 2019 and it looks like it was pinned, so I thought I was not getting newer versions.

I know how things work on this hosting provider because I worked there for 6 years, and I wrote a big chunk of the code moving their cloud. I know it is changing a field in the admin console, and that takes less than 30 seconds.

Unfortunately my ex-employer was acquired by a larger hosting company, they let go all the dev team, and no further changes are planned for the old platform; and that explains why my server was running an old kernel. Although I understand the policy, I don’t think it is a good one. If that kernel doesn’t work with Buster, I’m sure other customers are experiencing other problems.

There is a path to migrate the services to a current platform, and it will be forced to everyone at some point; but it is unclear when I can migrate my server. Not sure why is that, but it is possible is not a fully automated process.

Anyway, as an ex-employee, my server was completely free, so I think it is fair to move to some other provider and leave them alone. I’m very happy and grateful for having that free service, I wouldn’t mind paying for it –after all, I know we built a good platform–; but I’m not sure about this new hosting company.

Yesterday I finally migrated all the services I had left on that box, and I’m ready to switch it off. And it is a bit sad. In a way it is my last ties with a company I enjoyed working for, and doesn’t exist any more. And that includes all the code I worked on during those 6 years.

That’s OK. Software becomes legacy eventually, and it gets replaced. It is a cycle.

Getting distracted

Making a game isn’t easy. Focusing on a game until it is finished and released, is very hard. It looks like I have been successful at doing it for the last 8 years or so, releasing at least one new game every year –sometimes two!–. And, honestly, I don’t know for how long I will be able to keep it up.

I tweeted this two days ago, and it seems to have resonated with people:

What if the real happiness was starting projects, work on them as much as you feel like, and then never finish them?

There are those times in my life that I feel very productive, like I would like to work on so many projects, including a few that I know are not going anywhere. Early signs of that is when I started to work in a CRPG –this time for the CPC, and I’ve got further than ever before!–, or my latest work with Go and Ebiten.

And a lot of other projects I shouldn’t start because it is likely I won’t finish them ever –a compiler for a toy language? sure!–, and it all makes me lose energy, momentum, and focus on the projects that I could finish.

Hyperdrive started back in November 2021. It is a difficult one, that needed a few rewrites of the engine before I got it where I wanted, but we are almost in April 2022 and there is still a lot of work to do.

I guess it really sucks feeling productive, because I get easily distracted; even if that may be the real happiness.

A wiki and Tagless Final

I’m still refreshing my Go, and my friend Oscar is meeting Go for first time. Having a chat, I proposed him a possible project that could be interesting to have something to work on as he learns. Because learning to program a new language requires programming on that language, and for that, you most likely need a project.

I guess it could have been a blog as well –and I know it because I wrote my own blog engine in PHP, Perl and Python–, but I contributed a spec and Oscar is working on wikingo now based on that.

Looking at Oscar’s code, last weekend I thought: why not give it a go using Scala and some of the tools I work with at my day job but I never have time to do other things with?

So I wrote a toy wiki with Scala, http4s, doobie, ScalaTags and CommonMark (to render the Markdown), and using Tagless Final pattern –some context in the Expression Problem, and I haven’t found a “good” resource to link–.

Tagless Final is a pattern in functional programming, and is just an abstraction to build software, in which you define a Domain Specific Language (DSL) using Algebraic Data Types (or ADT, for example: a generic trait with abstract methods), and then provide an implementation (also called interpreter).

// my DSL
trait Wiki[F[_]] {
  type R = TypedTag[String]

  def init: F[Unit]
  def getPage(id: String): F[Option[R]]
  def editPage(id: String): F[Option[R]]
  def savePage(id: String, body: String): F[Int]
  def searchPage(query: String): F[R]
}

And this is relevant because http4s is built on top of Cats, which is a library for functional programming in Scala borrowing ideas from the Haskell programming language, and supports Tagless Final nicely.

Anyway, if what I have written so far doesn’t transmit the idea that is complicated, let me tell you: it is!

My expectations was to put together something quick and easy, but I got myself stuck in the same problems I have often encountered in my day job.

Some notes:

  • Missing documentation and examples –if you are trying to do something not in the docs, it isn’t easy to discover things yourself–. This was specially painful when trying to decode the form data: I was missing an implicit, and even looking at the examples, I didn’t see that my type F was asking for Sync but I really needed Asyncfacepalm–.
  • Because I decided to use H2 as database, I got stuck a couple of times with doobie –always my fault, if you use H2 as an in memory database, remember to set DB_CLOSE_DELAY=-1 in your connection properties, or your DB will be wiped mysteriously–.
  • I got stuck finding a mapping for Java’s LocalDateTime in doobie, even if I knew there has an implicit somewhere. The answer is deep down in the FAQ, instead of being included in the part of the docs that deals with mappings.
  • Turns out there aren’t many current Markdown to HTML libraries, CommonMark works fine –although it is a Java one–.

The end result looks very nice. Is not that different from a Model View Controller pattern: Wiki provides a description of the controller, with implementation in impl. It calls the models –implemented with SQL in WikiSql–, and maps to the views in WikiTemplates. WikiServer has the router that calls the controller. Concise yet very readable.

So I didn’t dislike the result, but oh it was hard to get there!

Re-learning Go

As I mentioned in you learn by doing it wrong, I was thinking that perhaps it would be a good idea to work on a CRPG targeting a modern platform, because it would be easier to fail faster than with my current approach with the Amstrad CPC.

I had started all the infrastructure to start writing code in C using SDL2, with the latest version compiled from source, and I was almost convinced that I wanted to use C. I even had researched some libraries that I would need to work comfortably. Being a modern platform, there’s no point on suffering the same struggles of making software for an 8-bit.

But then I remembered that, no so long ago, I was looking for a language that was higher level than C, compiled to native code, and ideally supported cross-compilation –at least let me build Windows binaries on Linux–.

I started looking at Haskell –because it would be also useful in the context of my day job, where I write functional code in Scala using ideas borrowed from Haskell–, but I got bored pretty quickly by the book I was reading. Then there’s that Rust language that the new kids are always talking about, but I have the feeling that it would be unlikely that I could use it professionally.

Then I remembered that I wrote a good amount of Go before moving to Scala –professionally–. Back then I thought Go was alright, with a few things I didn’t like, but it was mostly down to some tooling or the way the Go community handled releases that I found it wasn’t my cup of tea. At the end I was writing C at home, with a lot of Python for my tools, so I kind of left Go behind.

I’m sure I learned about Ebiten at some point. “A dead simple 2D game library for Go”, and after looking at it I thought it was neat. But I wasn’t doing Go any more, so I didn’t dig too much.

The language has changed since then, and it looks like some of the things I didn’t like are better now. So I have decided to revisit Go, and give Ebiten a go –the Go jokes never end–.

I have a plan and I’ve been streaming on my twitch what I have been doing so far. With the new shiny toys for vim –for example: LSP–, the experience is quite nice and I have enjoyed it so far –mostly, the interface bingo is still tricky if you don’t know the packages–. We will see how far I will get, because I didn’t do anything after refreshing my Javascript.

I’ve been asked if the code will be open source, to follow along as I stream my sessions. That has been a bit distracting, to be honest, because doing things in the open as open source requires some effort. I’m not sure how to do it, if making the engine open and then make a game with it –very ambitious–, or just keep things simple and make a game –which may be a feat on itself!–.

As I say: distracting. In reality I don’t think I will write anything too exciting.

Meanwhile, my progress on Hyperdrive is steady. The engine is solid, I have one stage –complete with final boss!–, and I’m working on graphics and enemy designs for the second stage. I will stream more sessions working on it, but only when the graphics are ready.

Breath of Fire IV

So after I gave up on Final Fantasy IX, I started playing Breath of Fire IV on the PlayStation, and it is a beautiful game.

I love the graphics –hand-drawn by Capcom–, the character design, how the story is told –e.g. you play a character that is not part of the party to know about his story–, the combat, how you can learn new skills from enemies –it is amusing learning the “command” special from the duck commander and use it to ask its minions to kill him–; there are a lot of things to love in this game.

A screenshot

Shame I don't have a better screenshot

So, why I am bouncing from this game after roughly 10 hours? Because I wanted to play a JRPG, and this one has a lot of mini-games that ask me to navigate screens to do uninteresting things.

And that is the main problem. The game looks fantastic, but the way they implemented the isometric graphics requires you to constantly rotate the camera to see where are you and where are you going. So having to do that chasing some sort of wild boar to find the mayor of a weird town so you get permissions to cross some tunnels is not only uninteresting but annoying.

The combat is alright, I found it to have the right amount of options and strategy, so I didn’t mind when I had to fight cute mice and roaches –big, strong ones– when I was already used to fight undead and zombies.

But when I start missing the combat because I spent one hour finding orphans playing hide and seek so I could continue my main quest, having to kill a few mice and roaches is too little when the game asks me again to play one of those mini-games.

It is a shame, but considering how little time I have to put on a game like this, I don’t think Breath of Fire IV is for me!

Python and LSP (II)

So shortly after I posted here my notes about my Python with LSP setup, I decided to give the neovim native LSP support a go. Mainly because I found that vim-lsc didn’t understand some of the messages Python LSP Server was sending and, although it looks like that didn’t have any effect, the error reporting was a bit distracting –even when I tried to disable it–. So I thought, how difficult would be using neovim’s built-in support?

Very easy, actually. I guess I’m not the only one not willing to burn all bridges in case I have to go back to use vim –that has been around long enough to make me think it will be there for ever–. There is no reason to think that neovim will disappear, or go in a direction I don’t fancy. My vim-lsc setup did work with vim, and in some cases I don’t have neovim available (or at least not at the minimal version required to enjoy all this).

But the fact is that I’m already enjoying some neovim-only features, like Telescope, so let’s go with full in with neovim for my LSP needs –at the end of the day, it is what I use for development–.

The changes to my previous configuration are actually very simple. First, I removed the vim-lsc plugin and added nvim-lspconfig, wrapped on a condition, so it doesn’t load if I use vim or the neovim version is not high enough:

if has('nvim-0.6.1')
    " lsp
    Plug 'neovim/nvim-lspconfig'
end

That should make vim –and older neovim– load fine.

Then, in my init.vim for neovim, I added an include to the file that will have the LSP specific configuration –and now I realise it should be wrapped on a version check, as well–. This doesn’t include my metals setup because nvim-metals doesn’t use the nvim-lspconfig framework –I need to investigate this, actually–.

I decided to include the configuration in lua instead of vimscript, because it is readable, and because the minimal example is using that language.

For full reference –for my future self–, this is the configuration:

-- for nvim lsp support

-- Mappings.
-- See `:help vim.diagnostic.*` for documentation on any of the below functions
local opts = { noremap=true, silent=true }
vim.api.nvim_set_keymap('n', '<leader>d', '<cmd>lua vim.diagnostic.open_float()<CR>', opts)
vim.api.nvim_set_keymap('n', '[d', '<cmd>lua vim.diagnostic.goto_prev()<CR>', opts)
vim.api.nvim_set_keymap('n', ']d', '<cmd>lua vim.diagnostic.goto_next()<CR>', opts)
vim.api.nvim_set_keymap('n', 'da', '<cmd>lua vim.diagnostic.setloclist()<CR>', opts)

-- Use an on_attach function to only map the following keys
-- after the language server attaches to the current buffer
local on_attach = function(client, bufnr)
  -- Enable completion triggered by <c-x><c-o>
  vim.api.nvim_buf_set_option(bufnr, 'omnifunc', 'v:lua.vim.lsp.omnifunc')

  -- Mappings.
  -- See `:help vim.lsp.*` for documentation on any of the below functions
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<C-]>', '<cmd>lua vim.lsp.buf.definition()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'K', '<cmd>lua vim.lsp.buf.hover()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gi', '<cmd>lua vim.lsp.buf.implementation()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gsh', '<cmd>lua vim.lsp.buf.signature_help()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<leader>rn', '<cmd>lua vim.lsp.buf.rename()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<leader>ca', '<cmd>lua vim.lsp.buf.code_action()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gr', '<cmd>lua vim.lsp.buf.references()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<leader>F', '<cmd>lua vim.lsp.buf.formatting()<CR>', opts)
end

local signs = { Error = "🔥", Warn = "⚠️ ", Hint = "✨", Info = "ℹ️ " }
for type, icon in pairs(signs) do
  local hl = "DiagnosticSign" .. type
  vim.fn.sign_define(hl, { text = icon, texthl = hl, numhl = hl })
end

-- Use a loop to conveniently call 'setup' on multiple servers and
-- map buffer local keybindings when the language server attaches
local servers = { 'pylsp' }
for _, lsp in pairs(servers) do
  require('lspconfig')[lsp].setup {
    on_attach = on_attach,
    flags = {
      -- This will be the default in neovim 0.7+
      debounce_text_changes = 150,
    },
    -- FIXME: shouldn't this be only for pylsp?
    settings = {
        pylsp = {
            plugins = {
                pylsp_mypy = {
                    enabled = true,
                    live_move = false
                }
            }
        }
    }
  }
end

Which is the suggested minimal configuration, plus the keybindings I use in metals –and I’m used to them–, plus a couple of small things, that affect my metals setup as well –e.g. the icons in the gutter to signal errors, warnings, etc; also something I need to investigate further–.

And that’s all, really. Seems to work fine, and all the noise from vim-lsc seems to be gone. I don’t know if the native neovim LSP support is doing anything differently, but at least is not reporting errors, so I guess… if I don’t know about it, it is perfectly fine.

This is still a bit work in progress and there are things I still don’t fully understand, but I will get there!

Python and LSP

I wrote a similar post back in April 2021 –on Gemini, that’s obscure!–, but I updated my main Debian box and something broke. Then I revisited it all after I dropped the few projects I still had with Python 2, and it all stopped working –did it ever work Python 3? Not sure–. So there you go, I did it again and I’m documenting it for “the future”.

I use vim-lsc –for now–, because it does a lot out of the box, and with almost zero configuration. Even now that I use nvim-metals for Scala –in neovim–, I have configured most bindings to behave like vim-lsc because I like it. I guess I could investigate the native neovim support for LSP, but I’ve been lazy. vim-lsc is good, but it doesn’t look like it is actively maintained, so take that into account if you want to try this.

The way I’m enjoying Python lately is with:

  • Python LSP Server –this is a community fork of the unmaintained Python Language Server–. This does a lot of things already, even if you only use this, it is a good experience.
  • Mypy –an optional static type checker for Python–, via pylsp-mypy. The code is checked on save.
  • Black –a code formatter; “Any color you like”–, via python-lsp-black. The code is formatted on save.

You can install all this in your local user using pip with:

python3 -m pip install python-lsp-server pylsp-mypy python-lsp-black

This is best with Python 3.8 or later. You may also want to ensure neither yapf nor autopep8 are installed, or black won’t kick in. But you can skip black all together and use yapf –it is also nice, just not as nice–.

I’m not always annotating my code with types –and I should–, specially in old scripts, but in general this is my setup. And all controlled via vim-lsc and this piece of configuration:

let g:lsc_server_commands = {
 \ 'python': {
 \     'command': 'pylsp',
 \     'suppress_stderr': 0,
 \     'workspace_config': {
 \        'pylsp.plugins.pylsp_mypy.enabled': 1,
 \        'pylsp.plugins.pylsp_mypy.live_mode': 0
 \     }
 \  }
 \ }

let g:lsc_auto_map = {'defaults': v:true, 'FindImplementations': ''}
let g:lsc_preview_split_direction = "below"

vim-lsc will start the LSP server only when we edit a Python file. I recommend reading its help to know what are the keyboard bindings to navigate the code, autocomplete, etc.

I hope this doesn’t break again any time soon, specially now that I’m not using the outdated and unmaintained Python LSP server!

Streaming killed the YouTube star

You have al the info at @reidrac is CODING, but basically I tried streaming a couple of my coding sessions on Twitch, and I liked it!

The way I’m looking at it is different than what I used to do in YouTube (and these are the videos so far): my “off-line” sessions are videos to keep in the channel, while in Twitch is more about sharing my programming session as they happen –currently I’m not keeping the videos, although I may keep some that are specially good to get a bit more attention–.

What I would like to get from these sessions? Good question!

First and foremost, finishig my current game –or whatever is the project I’m coding on, it won’t always be 8-bit gamedev–. If Twitch helps me with motivation and, in a way, to gamify the development process –like sharing progress on Twitter–, that’s great. One of the hardest things of gamedev is being persistent, because making a game is a long road, and until you get to the end… there’s no game!

Secondly, 8-bit gamedev is kind of lonely activity as in it is very unlikely I chat about it with anyone in “real life”, so I’m curious to see how will it work the community part of a streaming channel –which initially will be none, because nobody will watch my sessions–.

I was joking on the title of this post –see the Buggles song–, and I plan this to be non-stress and it will last as long as I feel like doing it. I won’t have the webcam and/or the microphone on if I’m not alone or it isn’t quiet enough around me –I don’t have dedicated office space at home–, and it is unlikely I will be able to schedule the streams in advance: I will code as I usually do, with the difference that I will be streaming it.

My Twitch channel is MrReidrac –because my reglar nickname was taken!–.