I got to a blocker with my TCP server using Akka Streams.

Looking for some examples, I found a few references to this piece in the Streaming TCP section of the docs:

Closing connections is possible by cancelling the incoming connection Flow from your server logic (e.g. by connecting its downstream to a Sink.cancelled and its upstream to a Source.empty). It is also possible to shut down the server’s socket by cancelling the IncomingConnection source connections.

Which sounds complicated, but after a couple of days of looking at the problem, it doesn’t make a lot of sense. Neither did for this fellow programmer 5 years ago (his solution is around the server, and I think the answer should come from terminating cleanly the stream).

And you know it is a tough problem when the only examples of TCP servers you can find online are pretty much the ones you have in the Akka docs –if this wasn’t a learning exercise, I think I would rather use a different technology–.

The problem I think is that when you are processing the request, you are in the flow, and is not possibleto do anything from there to complete the stream. Not sure if there is a reason to force this by design.

Sounds to me like a common pattern: client makes a request, the server sends back a response, and then it closes the connection. With Akka streams you can’t do that from the place where you are processing the request, at least as far as I can tell.

So it goes for ever, unless the client (or the server in case of an exception; for example an idle timeout) closes the connection.

My current code uses a Source.queue because it is supposed to complete the stream when complete() is called on the queue.

    Tcp()
      .bindWithTls(conf.address, conf.port, () => createSSLEngine)
      .runForeach { connection =>
        logger.debug(s"new connection ${connection.remoteAddress}")

        val (queue, source) =
          Source
            .queue[ByteString](1024, OverflowStrategy.backpressure)
            .preMaterialize()

        val sink = Sink.foreach { b: ByteString =>
          handleReq(connection.remoteAddress.getHostName(), b.utf8String)
            .map(_.compact)
            .map(b => queue.offer(b))
            }
            .run()
            .onComplete {
              case Failure(error) => logger.error(error)("Internal error")
              case Success(_) =>
                logger.debug("setting queue to complete")
                queue.complete()
            }
        }

        val handler = Flow.fromSinkAndSource(sink, source)

        connection.flow
          .idleTimeout(conf.idleTimeout)
          .joinMat(handler)(Keep.right)
          .run()
      }

The terminology of Akka is very specific, but basically I connect a flow to the incoming flow, created from a sink and a source. The sink reads the input and adds the response to a queue, that is where the source gets the data to send back to the client.

This seems to work. But the call to complete() is not completing the stream and closing the connection. If it change it for a call to fail, it does work by cancelling the stream.

And this is where I am at. I asked for help on Akka’s Gitter channel, without answer so far, but at least someone solved the issue with queues, so I must be doing something wrong!

Update: it is important that you read the second part.

It is Thursday, and this week most of my gamedev has been basically “admin”. But that’s OK because I’m very happy with my Gemini server –that just today got SNI support implemented–, so no complaints!

What I have today is mostly news regarding Brick Rick: graveyard shift.

First, the cover art by na_th_an (of the legendary Mojon Twins group) it is looking amazing already. I’m looking forward to see it in colour!

And secondly, the big news that the game will be published in the 499 range of tfw8b.com.

I know “The Future Was 8-Bit” for many years now. I’ve been customer –their hardware for our 8-bit machines is great–, and I like them personally. So I’m happy that Brick Rick is going to be available in cassette on a lovely budget range.

Is Graveyard Shift a budget game? I don’t know if that distinction makes sense any more in new 8-bit games (see that I didn’t say homebrew?), but I like the style and for this game I wanted to try something different to the big box releases I’ve had with Poly Play so far.

So that’s all for this week. Not a lot, but at least a good peek of what is coming!

As part of programming a Gemini server, I’m dealing with some classic problems, such as directory traversal attacks:

A directory traversal (or path traversal) attack exploits insufficient security validation or sanitization of user-supplied file names, such that characters representing “traverse to parent directory” are passed through to the operating system’s file system API.

For example, if the content we are serving is in /var/gemini/, our server should only serve content from that directory. In other words, the following request should be illegal:

gemini://server.local/../../etc/passwd

That file should never be served, of course.

There are different ways of preventing this type of attack, the most common is:

• Get the root of our directory, in this example /var/gemini/.
• Add the requested path, and notmalize the result; basically removing relative path components (for example, the ..). In this example we would have:

/var/gemini/../../etc/passwd -> /etc/passwd

• If the resulting path doesn’t begin with the root directory, then the path is invalid.

It is very straightforward. But there’s one thing I don’t like: it leaks directory information out of the root directory.

For example, let’s say this URL is valid:

gemini://server.local/../../var/gemini/

It translates to exactly the root directory, and that’s a valid path. Which I guess is not that important, but besides the fact that the server is leaking the root directory path, that URL can’t be easily normalized and we could have multiple valid URLs for the same directory.

So I thought of an alternative way, that detects that URL as invalid, and all the valid URLs are easy to normalize via a redirect.

The basic idea is to split the path by the / separator. Then calculate for each path component a value based on:

• .. goes back one directory: value -1.
• . is the same directory, so no change: value 0.
• Any other component: value 1 (as we go forward one directory).

If the value for a path component is less than zero, we can say that the path is illegal and we can happily return a bad request response.

Let’s show some Scala code:

def validPath(path: String): Boolean =
    !path
      .split('/')
      .drop(1)
      .foldLeft(List(0)) {
        case (acc, "..") => acc.appended(acc.last - 1)
        case (acc, ".")  => acc.appended(acc.last)
        case (acc, _)    => acc.appended(acc.last + 1)
      }
      .exists(_ < 0)

This code is called with the path already decoded from the URL, and the maximum length has been checked as well.

If the URL is valid, as in we never “went out of the root directory”, we can continue, and check as well if the URL is not normalized and redirect to the normalized version.

Now we can safely normalize and redirect. For example, all the following requests are redirected to the same path (the root document):

gemini://server.local/./
gemini://server.local/directory/../
gemini://server.local/directory/another/../../

I found it all very interesting, despite being an old problem to solve.

What is the Gemini protocol?

Gemini is a new, collaboratively designed internet protocol, which explores the space inbetween gopher and the web, striving to address (perceived) limitations of one while avoiding the (undeniable) pitfalls of the other.

In my opinion, the Wikipedia page is more informative (if a bit technical):

The Gemini protocol is an application layer protocol for distributed hypertext information systems that provides access to simple, primarily textual documents in Gemini space. This is done with contemporary technologies such as TLS, thereby improving privacy and user agency relative to the Web. The protocol is being designed collaboratively and is not currently being standardized as an internet standard.

According to that wiki page, the project is quite new, even for Internet standards, as it was introduced in June 2019.

Last week I was looking for a project to tinker with fs2 streams, so I thought that may be writing a simple Gemini server could be perfect.

That didn’t go well, and I ended frustrated and rage-quiting fs2, but that’s a story for a different post. So I thought, what about Akka Streams? It is a completely different beast, but the docs are better and there is no need to deal with the tagless-final pattern, so I found the API quite pleasant to use.

After a couple of days of coding, I’m getting close to beta quality, and I have learnt a lot in the process.

It was inevitable that I went down the Gemini rabbit hole, and I tried some clients:

• Lagrange: probably the more flashy client, although the SDL rendering makes it feel like a weird desktop app.
• Amfora: a console client. Considering that Gemini is text-only, this one is excellent.
• Ariane: this one for Android. It is probably not as polished as the other two, but is nice to have Gemini in your phone I guess. (Edit 2022-05-17: this client is no more).

What I found in Gemini really surprised me.

It feels like an early Internet, as in a smaller place, with reminiscences of webrings from the 90s. There is less noise and more focused content. I’ve found forums, personal pages, and blogs.

The use of RSS is widespread and looks like is one of the main ways to discover content via aggregators (although there are at least two search engines). Some hosts provide mirrors and proxies to the world wide web, as a simplified version of the same content.

And because the name of the project comes from the second U.S. crewed spaceflight program, it all has its own flavour: pages are capsules, blogs are gemlogs (or glogs), and there is a CAPCOM (from capsule communicator) and a Spacewalk (both aggregators).

I tried one of the clients and, without thinking about it, I ended spending a good time reading random blogs. I mean, glogs.

I plan to continue with my server and, probably, release it at some point when I think it is production ready. And, who knows? I may put some content on the Geminispace.

It looks like it’s Thursday again, and that means gamedev update!

I spent probably too much time to get this blog and the website in order. Mostly getting the light/dark themes to behave, some CSS here and there, and after torturing a bit my Hugo templates, I think it is “perfect™” now.

And there was some time for gamedev as well, of course.

Brick Rick: Graveyard Shift

I finished the music, and posted a WIP of it. The music is not exactly like that any more, that’s the WIP spirit right there.

There was more testing. My good friend Antxiko is doing a great job and found some behaviour that had to be improved in the pause code, and that’s done. We will keep testing a bit more, although at this point I feel like playing the game just for fun because this is “release candidate” quality already.

The cover artist provided a first draft, and I love it. I’m looking forward to have something final so I can show it.

I’ve been told that it wasn’t intentional, but to me the cover art feels a bit less cartoony than the Amstrad CPC cover. The sprites look less chibi on the speccy, and I changed the style slightly, so it could be that or just a happy accident.

I had a chat with the other artist that is going to work on the loading screen, to double check what is needed. At the end it will be better if he has something that is as final as possible as it will be in the inlay (including the lettering of the title).

I’m not really in control of the times, but I suspect we could be 2 or 3 weeks away from the release. Watch this space!

Better Windows support in ubox MSX libs

~mk~, user of MSX Resource Center, has been trying to make the ubox MSX libs compile on Windows.

After some back and forth –I don’t have a Windows system to test the changes–, it looks like he managed to compile the libraries, the tools and the example game. Great success!

I released version 1.1.4, once again without testing the changes myself. At least they don’t break anything on Linux, so perhaps this is the release that solves all the problems on Windows (or it only requires minimal tweaks).

And that’s all for this week in gamedev.

This Saturday (Feb 13th) I will be attending (virtually) to Scala Love in the City, a conference for Scala developers.

As you can see, the schedule is very promissing. It all starts at 8:00AM GMT, which is not bad at all!

It’s been a while since the last time I attended a conference, and I mean pre-COVID times, but I usually watch some of the talks when they’re uploaded to YouTube or similar. This time I’ll try to attend in real-time. Even if it is online, I’m sure there will be a difference –I guess you can ask in the Q&A time, and there will be party–.

Personally, I’m interested in any talk that focus on Scala 3, and of course the keynote by Martin Odersky.

Any novice programmer that decides to make a game, usually has an oversized type of project that would like to tackle, specially as first project.

In my case it was an CRPG (Computer Role-Playing Game), and it is funny to see how things change a lot to essentially remain the same. I’ve seen newbies trying to make their first game, and it was a JRPG (Japanese style RPG). Then an MMORPG (Massively Multiplayer Online RPG). At some point, interest shifted to the FPS (First Person Shooter) genre. The White Whale changes, yet remains the same, unreachable.

Now that I’m more experienced and I’ve finished a few games, perhaps is time to make that CRPG; although not even now I’m sure that I’m capable of finishing such a project!

I’ve spent some time deconstructing the genre, and I read “Fundamentals of Role-Playing Game Design” by Ernest Adams, a bit by accident. I can’t remember when I bought the ebook.

Although the book didn’t tell me anything new, strictly speaking, it was a good overview of the genre, and it kind of encouraged me to give the idea another go.

At the end, I think I’m going to focus on the core gameplay, and then expand from there. Instead of having everything I think a CRPG should have, I’ll see how much of a subset is sufficient to recognise the genre.

I’ve settled in a few points:

• It will be dungeon focused, probably with no over-world or “city” view.
• As part of the dungeon exploration, there will be object/key/switch puzzles.
• I’m keen on a rogue-like component, so I don’t have to design the dungeon –and I can have more fun playing it–.
• There will be inventory, and some sort of management of it.
• I’d like it to be story heavy, or at least with an interesting story, via scripted scenes and NPC dialogue.

The only part that I’m undecided is the combat. I know the player will walk into a new room and there will be enemies, and there will be a fight.

For now I have this list of possibilities I like:

• JRPG turn based combat, party based. Limited actions: attack, magic, use item, defend, etc. See Bard’s Tale, FF or Dungeon Quest series.
• Turn based combat, party based and the environment matters. Same actions as before, but the characters move on a map that can be used strategically. See Ultima or Shinning Force series.
• Action RPG, with little skill involved. The player bumps into the enemies, and stats kick in. See Ys, and lots of rogue-likes use this mechanic.
• Skill based action RPG. Which is not really, really an RPG, but I guess you can make stats count. See 8-bit entries in the Zelda series.

Depending on what I choose, the implementation may vary a lot. For example, inventory management is quite different if you manage one single character or a party.

Another constraint is that it must be small, even if that means being less of an CRPG. Mostly because that would increase substantially my chances of finishing it.

I can start working on the points I know for sure I want in the game, and hope that, as I work on them, the rest will reveal itself naturally.

I have some nice Zelda-esque tiles for the MSX, so I may have decided the platform already!

Brick Rick: Graveyard Shift is getting there!

In this WIP video I’m testing the music, that reminds a bit of my own Night Knight, and perhaps some Ghost and Goblins –which is not a bad thing–. Graveyard Shift will need a ZX Spectrum 128K model, and the music uses the AY-3-8912 audio chip.

Looks like I didn’t do a great job capturing the sound in this video. The emulator was producing stereo (although the ZX Spectrum wasn’t capable of that), but the captured video is mono. Hopefully, the next video will be better.

I’m looking forward to the release!

Reading Continental drift by Jon Udell:

The web of the late 90s was a cornucopia of wonder, delight, and inspiration. So was the blogosophere (sic) of the early 2000s. I know a lot of us are nostalgic for those eras, and depressed about how things have turned out. The bad is really quite bad, and sometimes I feel like there’s no way forward.

Also recently on HN: Turn to RSS Feeds to Regain Control of the World Wide Web (too often Hacker News comments are even more interesting than the story).

There’s a good summary at the Wikipedia, with some notes on current usage (all negative, btw).

Summing up:

• RSS is not dead, but is not fashionable (and arguably, it has seen better times).
• Social media is hostile, and major sites such as Twitter and Facebook have removed (or reduced) their support; some of them never supported it.
• All major browsers have removed RSS support (IE still supports it out of the box; and Firefox, champion of the Open Web dropped support of RSS in Firefox 64).
• All the problems it had, are still there (probably for another post).

Although I like Udell post and his optimism because we still have the Internet, and despite how things are going and “we’ll figure it out”, I feel something must change if we want to get back to that cornucopia of wonder, delight, and inspiration (and I want the blogosphere back!).

As bonus: in defense of RSS by Seth Godin, written 10 years ago and updated just recently. Still as relevant now as it was back then, if not more.

As part of my exit from Twitter, I plan to have some work in progress (WIP) posts and information on what I’m doing in my “gamedev” projects. Or at least, updates mostly related to game development, because I think I may include other development related topics, but gamedev is catchier than plain dev, isn’t it?

This is not Twitter. The format is different and, essentially, even if I make notes as the events happen, I don’t see much value on posting them “as-is” (although people liked that on Twitter).

So instead I’m going to write one post a week with those notes more or less redacted, although I may still post specific WIPs any time when the content is longer than a short Twitter thread.

Looking at my usual week, the updates are likely to be from Thursday to Thursday, but this is not a rule set on stone anyway.

Let’s do this!

Brick Rick: Graveyard Shift

Not a lot has happened on this project. Basically: the game is finished!

I said back in December last year that it was very likely that the game would be out by the end of January 2021. So, what’s going on?

First of all, I started with the struggle of drawing the loading screen, because it’s not my thing! Then I remembered someone contacted me in April 2019 offering a collaboration to draw the loading screen of a game I was working on back then –a game that was cancelled, by the way–.

At that time I was impressed with the screens that he had produced until that point, but I politely declined because I had that project covered already.

Anyway, the good news is that he will be working on the loading screen of Graveyard Shift; although he asked for some reference art, and I don’t have that. Which makes sense, if I could draw, I would draw the loading screen myself.

The truth is that I didn’t want to think about a physical release at this point, but then my wife suggested (wisely, I must say) that I could commission the cover art and use that as reference for the loading screen. I’m happy to say that I’m working again with the artist that did the cover art of Brick Rick for the Amstrad CPC.

Now that I’m going to have cover art, almost accidentally I have found a publisher that is keen to release the game on cassette in a nice budget-style label. It’s not a closed deal, but I’ll move it forward as soon as I have the cover art.

Looks like things are coming together nicely, but all this takes time.

Meanwhile, testing is going very well. A few bugs were fixed in the first pass –some of them of the embarrassing type–, and I think we already have a version that we could call “release candidate”.

There is still at least one song missing, and I should tackle that before the art is ready. Turns out, the Halloween-ish theme isn’t that easy.

I don’t like that the project has gone a bit cold, but I don’t have a deadline anyway.

The website of the game has been online for a while now.

Drawing on the MSX

After I released my libraries to make MSX games in C, I got the MSX itch again. I guess I had lot of fun putting together the example game in 3 days or so.

I’ve been drawing some tiles, partly looking for inspiration, partly aiming to a game idea that perhaps could work fine (spoiler: with a rogue-like component).

It is still too early to reveal anything, but I’m excited. I may even consider submitting the game to MSXdev'21, although I haven’t been much of fan of game jams in the last few years.

Refactoring my Scala.js and Canvas 2D experiment

I’ve been working on and off on my Scala.js and Canvas 2D experiment for a while now, and I don’t know what to do with it.

A nice side effect of not knowing what to do is that I’ve been reviewing the code again, and after this last refactoring, I think it is starting to look great.

In some parts I have decided that it doesn’t make much sense to go full FP because of the interaction with the Javascript underneath. A bit of Object Oriented Programming (OOP) is just fine.

The idea was to:

• write some Scala out of my day job
• make gamedev targeting the web more comfortable by using the same tools I use working with Scala
• have some fun!
• may be make a game?

So I’m fine with that (although to be clear: the code was fine already).

And that’s all for this week in gamedev.