BAF's Ramblings » #bafirc

BAFIRC: DCC Layer

October 10, 2010bafirc

So, I spent some more time working on BAFIRC. I ended up beginning the DCC layer, as it doesn’t seem to be too much of a jump over the CTCP layer. So far, I only have a similar infrastructure as CTCP has, receiving/sending and parsing the requests. I wrote some quick hacked up code for connecting to a DCC CHAT, however I still need to write more robust classes that will handle the connection layer and handle transferring files. This will mostly be a bunch of brain-numbing, boring code to write, so I may put it off, not sure yet. I am also debating on whether or not to implement an ident server. I’d like to run the core on a Linux box at some point, under Mono, but right now, Mono doesn’t have a stable release supporting .NET 4, so it will be Windows-only until that happens (unless you’re feeling adventurous, I hear that the current SVN head of Mono has full .NET 4 support). Obviously, on Linux, it an internal ident server won’t be used (for a couple of reasons I won’t get into here). I still think it will be beneficial to create, so I may whip one of those up - but again, that falls along the same lines as far as being mind-numbingly boring code to write.

Moving on to more interesting topics, I really need to plan out my plugin system. As I’ve said before, plugins will do just about everything, so it will be important to design this correctly. I’m going to use MEF composition (new to .NET 4) - so I just need to figure out the best way to expose the plugins. One way of doing it is to create an interface, or an abstract base class, with methods for all of the various events that may be called, and then just call them on all the plugins. Another option is to have metadata telling the plugin manager what to call and when/why to call it. However, neither of these approaches lines up with the whole Observable pattern I’ve been playing with and growing fond of. It seems like, in order to utilize that, I will simply require some sort of initialization function in each plugin that can then attach and subscribe to the various Observables it’s interest in (or ones that it derives from the base ones). My only concern with this approach is that it doesn’t seem too OOP-y. I think I can make it work though. One other thing I may support is some sort of permissions/security system for plugins - those of you who have installed extensions in Chrome or have installed applications on Android know what I’m talking about - the screen that shows you what permissions the extension/application requires before it’s installed/activated/whatever. You shouldn’t be loading potentially untrusted plugins into BAFIRC anyway, but I figure it will be a nice touch to show what sorts of messages the plugin hooks in to, and then restrict the plugin to using only those. I’m thinking of a manifest-like metadata attribute on the plugin, and then on plugin initialization, it sends this manifest to the plugin manager, which then can check that it’s been granted all permissions it’s asked for (or prompt/warn user if not), and then return an interface to only those event observables. This is still a fairly early and under-developed idea, but it’s a start (and it’s more than I had when I began writing this post)!

Anyhow, I’ve built another wall of text, so it’s probably time to sign off for now.

BAFIRC: CTCP Layer Refactoring

October 9, 2010bafirc

So, I’ve refactored the CTCP interaction in the core IRC connections - mostly to just neaten things up and do them in a way I feel is much more cleanly done. This didn’t change functionality at all.

I also added some helper methods to get IObservables for the main IRC events, which just wrap the existing event handlers. This is purely added functionality, allowing .NET RX to be used much more easily, rather than having to manually calculate the observables from the events yourself. There are also some extra methods that allow filtering on command type (so with one function call, you get an observable for only one type of message (e.g., privmsg) that you can then subscribe to, or filter further).

To go along with this, I did end up creating another CTCP layer that sits on top of the IRC layer. The core layer still differentiates between regular privmsg/notice and ctcp/ntctp, and the CTCP layer just extends the functionality by making the data easier to access, reply to, and work with. Also featured in this layer are some extension methods that allow you to get CTCP events - notably CTCP received and CTCP sent, along with some extra filtering options (mimicking those in the irc layer).

These changes are mostly minor, but they do enable you to do more powerful event filtering easier and with less boilerplate code. For example, before these additions, in order to reply to a CTCP PING, you would have to write the following code:

// given IrcConnection conn;
conn.MessageReceived += (s, e) =>
{
    if (e.Message.Command == IrcCommand.Ctcp && e.Message.Parameters[1].Equals("ping", StringComparison.OrdinalIgnoreCase))
        conn.SendMessage(IrcMessage.NCtcp(e.Message.From.Nick, "PING", e.Message.Parameters[2]));
};

With these additions, responding to a CTCP ping is simple:

var pingRequest = conn.GetCtcpMessageReceived(CtcpMessageType.Request).Where(request => request.Command.Equals("ping", StringComparison.OrdinalIgnoreCase));
pingRequest.Subscribe(ping => conn.SendMessage(ping.CreateReply(ping.Data)));

The main strength, at least from what I’ve seen so far, is that you gain the ability to filter and join events using LINQ. So in this case, we filter down a CTCP Request (and, the GetCtcpMessageReceived() function just filters the more generic MessageReceived event from the irc layer, using LINQ) down to a PING request, and then subscribe to that specific event, and handle it by sending an appropriate ping reply.

I will definitely be exposing all events to the plugins like this. Depending on what turns out to be easier, I may keep the same event model for firing events, but the Observable wrappers over these events are very powerful and make the code much more concise (just like LINQ does).

Note that the LINQ code is, as most LINQ code usually ends up being, much more dense. I usually tend to split things up over lines to ease in readability, and help keep individual lines short (I usually create a new line at each part of the chain). There’s also the LINQ syntax, which looks more like SQL than C#, which is just turned into these chained function calls by the compiler anyway. The LINQ-ified code can be harder to read, especially when you’re not familiar with LINQ, but I find it tends to be more result-oriented. Rather than looking at a bunch of if statements and crap, you’re being much more declarative in what you want. Rather than saying “give me everything, and if some condition holds, do this,” you’re saying “I don’t care how you do it, but give me things for a certain condition, and then do this.” The example I gave above was trivial, but I’ve seen some much more complex examples involving sequencing that make the LINQ code much easier to write and less messy - such as this.

I got way off on a non-BAFIRC related tangent, but LINQ is awesome. It’s one of those things that seems questionable or even useless, but once you start using it and thinking in terms of ‘how can I do this with LINQ’ it becomes much more useful. I find that it encourages you to think in abstract steps, rather than focus on how to actually implement what you’re doing, and then you can stay at those abstract steps and let LINQ figure the rest out.

BAFIRC Progress

October 9, 2010bafirc

I was lazy today and didn’t do any work on BAFIRC. I did start looking at the new .NET 4 features, on a tangent, while I was refreshing myself on MEF (I’ve used MEF before, and now it’s integrated in .NET 4).

One new feature I stumbled upon is the new event system, with IObserver/IObservable and the fact that these can be push or pull events. It’s described as being LINQ for events. I’m considering refactoring my IRC layer to use the new event model, though I need to do some more research. It seems really powerful and neat, with one line of code, the consumer can subscribe to only private message events, for example. This should cut down on boilerplate code in event handlers to pick and choose what to respond to.

BAFIRC: CTCP Layer

October 8, 2010bafirc

So, I was working on the CTCP layer tonight. I had planned to keep it a separate entity, and then debate ensued over how to actually implement ‘CtcpMessage’ - whether it should subclass IrcMessage, or be composed of it. After a lengthy discussion, I decided to just add it into the core IRC layer, with two new pseudo IRC Commands - CTCP and NCTCP, CTCP being for requests/extended data messages (like ACTION) sent via privmsg, and NCTCP being for replies sent via NOTICE.

I’m not totally sure if I like how I actually implemented this, I’ll have to sleep on it and see in the morning. I created a CtcpHelper class that looks at IrcMessage objects and maps back and forth the commands. When parsing a message, it checks if it’s actually CTCP, and if it is, forges the command type appropriately (it also rebuilds the parameter list, privmsg/notice only have two parameters, a destination and a message, but CTCP/NCTCP have three, destination, ctcp command, and ctcp data). This isn’t too bad.

The part I’m still not totally sure on is constructing it back to a string that can be sent to the IRC server. In here, I added another path to my if statements that maps the command back if it’s a CTCP message, and then when I actually print the parameters out, it calls another function in CtcpHelper to “unbuild” the array, dumping it back to destination/message and formatting the message into CTCP format.

Anyway, this is probably about as clear as mud. I’m pretty tired right now, and I was even making some stupid mistakes while coding. I’m going to go to bed and sleep a good long while (until I wake up, rather than until the alarm wakes me). If need be, I’ll revise or rewrite this post tomorrow.

BAFIRC: IRC Layer Complete?

October 6, 2010bafirc

So, I think the IRC layer is pretty much complete at this point. I’ve added options to enable/disable auto-reconnect, added error reporting/handling, and added configurable message throttling/flood control (with defaults that match the suggestions in RFC2813).

I did come across one bug while testing the throttling, where messages were being sent out of order. This was a bug in the way I was inserting messages into the prioritized queue. Just one character made the difference - I was determining the index at which to insert by counting the number of messages with a priority greater than the message being sent. What I meant to do was count the number of messages with a priority greater than or equal to the message being sent. The result was that when you sent a message, it was put in the queue in front of any other message with the same priority, instead of behind them all. Adding one tiny little equals sign to the code fixed that one.

As far as throttling goes, it is accomplished with a rolling timer. The connection processor is signaled that there is data waiting in the send queue by an event that is waited on along with the incoming data. Before, it would just dump the whole queue to the server whenever this flag was set. With the flood control now, it will dump one message at a time, until the throttle has been hit. At that point, it makes note of when sending can be re-enabled, and leaves early, before sending everything in the queue. On the next pass, if sending shouldn’t be re-enabled yet, we block the send-queue-has-data flag, such that we will only respond to incoming data. When throttling is occurring, a wait timeout is observed, so we can loop back around and re-enable the send-queue-has-data flag. Once this is re-enabled, we will hit the flag (if data is waiting) and keep sending data and repeating this process until the queue is emptied out. When sending is enabled, we revert to an infinite timeout, waiting on both receiving and sending.

There are three configuration values for the throttling - enabled, burst limit, and time per message. You are allowed to a number of messages, up to the burst limit, before you begin being throttled. After then, you can send one message every (time per message). The default values are derived from the algorithm described in RFC2813, section 5.8 - burst limit of 5, and time per message of 2 seconds. So, unless overridden (or disabled), you will be allowed to send 5 messages without being throttled (assuming you haven’t sent anything for the 10 seconds preceding that). Once you begin being throttled, a message will make it out to the server roughly once every 2 seconds (give or take a few ms, of course, depending on the OS scheduler, timer accuracy, etc.). Actually, I lied a tiny bit, it doesn’t actually wait 2 seconds per message in this case - with a burst allowance of 5, 5*2 = 10 seconds, so it will wait until 10 seconds after the first message in the ‘flood’ was sent (then 12, then 14, etc).

Error reporting and handling was added as well. If any uncaught exceptions are thrown on the message processor or connection processor thread, they will be swallowed and reported via an event. One change I may make later is to put some limits on this, that is, if you receive a configurable number of exceptions within a given time frame, then you give up and stop trying to reconnect or process messages (rather than potentially hammering the server if you keep being disconnected for some reason). I could also implement back-off into this, whereas it will try to reconnect instantly at first, then slowly back-off and wait longer and longer for a reconnect, until it finally gives up.

At any rate, I think that about does it for this portion of the project. It did run all night last night and all day today without disconnecting or throwing any exceptions, and I’ve tested out the throttling algorithm. SSL connections work. Everything seems to work great, so it’s time to move on. Still need to decide what I’ll start next… CTCP/DCC support or the plugin system. Decisions, decisions.