Athena | V21 | continuation of Kepler (C version) [C17, SQLite, libuv, NNG]

[Ewout]

Athena is a continuation of the legacy C version of Kepler, that fixes all threading issues and ports over functionality from the Java version and Havana to make it feature complete. It'll be targeting client revision 21 from '08 initially. It is multi-platform, and will compile and run on pretty much anything: though, the recommended platform will be Linux.

The internal multi-thread handling will be revamped by relying on isolated threads that communicate via intra-process message passing instead, eliminating the need for lock-based synchronization.

Libraries

• libuv
• libsodium
• NNG
• Collections-C
• stumpless
• SQLite (LumoSQL)

Objectives

• Port over all new features from Havana and Kepler
• .. basicly implement a feature complete V21
• Actor model inspired approach to internal thread-to-thread communication
• Packet cache similar to Snowlight

Screenshots

.. no. Can't even compile the damn thing yet!

Source code

https://github.com/oldhabbo/Athena

 

[Quackster]

Thread approved, good luck! Please format your development thread properly, add screenshots etc. I will add section rules at some point, but soon nonetheless.

 

[Near]

Go rock it.

 

[hackademiC]

Good luck with the development, keen to see the progress

 

[Ewout]

Currently in the progress of updating all library references to their new APIs.

Collections-C was never used correctly within Kepler (C version). It's designed to be used as OS-wide system library installed via package managers, not as a forked, modified locally scoped version. Need to create an AUR package first and modify the CMake configuration to be able to use it.

Update: created an AUR package for Collections-C here.

Update #2: created an FR for proper release versioning inside Collections-C and an RFC for better distro specific package availability.

Update #3: maintainer of the library commented and will implement a proper versioning scheme for the library.

Note: In the future, I'll try out the more performant liblfds and replace Collections-C with it if it satisfies all requirements.

 

[Pitt]

As I can see, this forum is motivating everybody :O
Really nice project Ewout ! Woot woot !

 

[Ewout]

Okay, got it to compile. Now discovering new C17 behavioral changes. It doesn't like how Kepler (C version) used global variables and no longer links as expected.

Spoiler: Console output

/usr/bin/ld: CMakeFiles/Kepler.dir/src/shared.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/util/stringbuilder.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/util/configuration/configuration.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/communication/messages/outgoing_message.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/communication/message_handler.c.o:(.bss+0x0): multiple definition of `message_requests'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x80): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/communication/message_handler.c.o:(.bss+0x13880): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/server/server_listener.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/server/server_listener.c.o:(.bss+0x80): multiple definition of `message_requests'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x80): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/server/rcon/rcon_handler.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/server/rcon/rcon_listener.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/util/encoding/base64encoding.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/util/encoding/vl64encoding.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/db_connection.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/pathfinder/affected_tiles.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/pathfinder/pathfinder.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/catalogue_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/messenger_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/moderation_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/player_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/items/furniture_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/items/item_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/rooms/room_favourites_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/rooms/room_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/rooms/room_rights_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/rooms/room_user_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/database/queries/rooms/room_vote_query.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/game_thread.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/moderation/fuserights_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/player/player.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/player/player_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/player/player_refresh.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/navigator/navigator_category_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/items/item.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/items/item_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/items/items_data_parser.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/inventory/inventory.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/tasks/roller_task.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/tasks/status_task.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/tasks/walk_task.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/room.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/room_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/room_task.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/room_user.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/public_rooms/pool_handler.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/public_rooms/walkways.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/mapping/room_map.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/mapping/room_model.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/mapping/room_model_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/mapping/room_tile.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/manager/room_entity_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/manager/room_item_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/room/manager/room_trade_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/messenger/messenger_entry.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/catalogue/catalogue_item.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/catalogue/catalogue_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/catalogue/catalogue_page.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/Kepler.dir/src/game/texts/external_texts_manager.c.o:(.bss+0x0): multiple definition of `global'; CMakeFiles/Kepler.dir/src/main.c.o:(.bss+0x0): first defined here

Going to cut out the global variables everywhere, and introduce a custom DI solution to resolve.

Note: can't seem to find any library for dependency injection in C, help in search welcome. I prefer adopting a more known library to rolling my own.

 

[Ewout]

...

Update #2: created an FR for proper release versioning inside Collections-C and an RFC for better distro specific package availability.

...

For Athena to be easily compilable by end users on Linux platforms in the future, I need Collections-C to be available in all distro specific package managers by default.

This'll be a long road and probably take a few months to years. The first step is getting distro packagers aware of this library and the need for it.

I have created an AUR package as first step, once this gets enough votes; it'll be able to get included in the community repo by a TU (Trusted User). From there it'll be easier to trickle down to other distros.

Consider registering on the AUR and voting on this package to help in this effort.

To solve the captcha, boot Arch Linux online from your browser here. Use the second window/textarea to paste the command.
Note: Let me know if this doesn't work, haven't tested it.

 

[Ewout]

Still altering the code, so it'll compile with a C17 compiler. Which means cutting out global variables for one, in the progress of reworking all structures in the code to accept IoC-like containers.

As well as figuring out how to cut out all the header mess, currently trying out makeheaders and following Richard Hipp's (author of SQLite) advice.

.. and PascalCasing all structs to feel more like OO classes, similarly to how C-Collections does this.

Update: for makeheaders to work properly, the codebase needs to be contained within a single root directory. A large scaffolding/rewrite it is!

 

[Ewout]

Okay, I spent about two days packaging up the new LumoSQL, which is SQLite modernized with a faster key-value backing store (LMDB). It took that long, as they decided to reinvent the wheel by implementing their own C build system in Tcl and had to figure out how that worked.

In the long run, definitely worth the effort. Athena will use it from now on, as one of the first projects running the fastest SQL database on earth.

 

[Ewout]

Replacing log.c with stumpless for logging. A more robust library that supports various targets.