docs/tech/1.html

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Style-Type" content="text/css"/>
<style type="text/css">@import "style.css";</style>
<title>IRC Services Technical Reference Manual - 1. Program structure overview</title>
</head>

<body>
<h1 class="title" id="top">IRC Services Technical Reference Manual</h1>

<h2 class="section-title">1. About this manual</h2>

<p class="section-toc">
     1-1. <a href="#s1">Intended audience</a>
<br/>1-2. <a href="#s2">Structure of this manual</a>
<br/>1-3. <a href="#s3">History of Services development</a>
</p>

<p class="backlink"><a href="index.html">Table of Contents</a> |
<a href="2.html">Next section: Core Services functionality</a></p>

<!------------------------------------------------------------------------>
<hr/>

<h3 class="subsection-title" id="s1">1-1. Intended audience</h3>

<p>This manual is written with an intended audience of programmers looking
to change or extend the IRC Services source code, and covers the overall
structure of the code as well as details on each of Services' functions.
The content of this manual is highly technical, and requires a reasonably
high level of expertise (both in programming in general and in the C
language and Unix computer systems in particular) to peruse.</p>

<p>For information on installing and using IRC Services, please see the
<a href="../index.html">user's manual</a>.</p>

<p>Note that this manual describes Services for IRC Networks version 5.1.24,
and its contents may not be applicable to modified versions or derivative
programs.  Also, this manual is intended to be used alongside the source
code; details of function parameters, structure fields, and the like are
often omitted for the sake of brevity and clarity. <!-- Use the source,
Luke! --> </p>


<p class="backlink"><a href="#top">Back to top</a></p>

<!------------------------------------------------------------------------>
<hr/>

<h3 class="subsection-title" id="s2">1-2. Structure of this manual</h3>

<p>Services can be broken up into two parts: the <b>core functionality</b>,
which implements the "IRC server" part of Services, and <b>modules</b>,
which add the "bot-like" functions to Services as well as provide common
interfaces to differing implementations of some functions.</p>

<p>The core Services functionality is primarily discussed in
<a href="2.html">section 2</a>.  Two topics, however, are given their own
sections due to their complexity: communications (<a href="3.html">section
3</a>) and module handling (<a href="4.html">section 4</a>).</p>

<p>Modules are divided into four groups:</p>

<ul>
<li><b>protocol modules</b> (<a href="5.html">section 5</a>), which
handle the interface between the generic IRC server implemented by the
Services core and the particular protocols used by various IRC servers;</li>
<li><b>database modules</b> (<a href="6.html">section 6</a>), which handle
the storage and retrieval of data used by pseudoclients and other Services
functions;</li>
<li><b>pseudoclient modules</b> (<a href="7.html">section 7</a>), which
implement the various pseudoclients themselves; and</li>
<li><b>other modules</b> (<a href="8.html">section 8</a>), including all
modules not covered above.</li>
</ul>

<p><a href="9.html">Section 9</a> covers <tt>convert-db</tt>, the tool
included with Services to convert databases from other Services-like
programs to an XML data stream that can then be imported into Services
itself.  The compilation of Services (the <tt>configure</tt> script and
<tt>Makefile</tt> files) is described in <a href="10.html">section 10</a>,
and areas where Services could potentially be improved are discussed in
<a href="11.html">section 11</a>.</p>

<p>A source code reference is included as <a href="a.html">Appendix A</a>
(by source file) and <a href="b.html">Appendix B</a> (by function name).</p>

<p class="backlink"><a href="#top">Back to top</a></p>

<!------------------------------------------------------------------------>
<hr/>

<h3 class="subsection-title" id="s3">1-3. History of Services development</h3>

<p>Before discussing the details of the Services code base, it is worth
mentioning the history of Services development.</p>

<p>Originally, I (Andrew Church, the developer of Services) started writing
Services as something of a "quick hack" for my personal server, which I had
set up for the use of a channel I frequented at the time that had decided
to move away from the DALnet IRC network.  DALnet did not at the time&mdash;and
to the best of my knowledge still does not&mdash;publish the source code to
their nickname and channel registration services, nor was I able to locate
any other similar usable open-source software, so I took the route of
writing my own.  This motive had, at least initially, significant influence
on the user interface design; the service nicknames NickServ and ChanServ
seem to have been around since the early days of IRC, but the basic
commands, such as <tt>REGISTER</tt>, <tt>SET</tt>, and the NickServ
<tt>RECOVER</tt> and <tt>RELEASE</tt> functions, were based on those seen
on DALnet.  (Ironically, it seems that DALnet has since incorporated at
least one feature, the channel successor concept, from this program due to
user pressure.)</p>

<p>The first "release" of Services&mdash;using the term loosely,
since I had no interest in releasing Services publicly at this
point&mdash;was made on April 5, 1996, when the program first went online.
Since my IRC server was mainly for private use, there was little call for
additional features, and aside from occasional bug fixes, not much
development was performed during this time.  The code base itself could be
called "functional" at best; while I have always put effort into writing
good code rather than simply "working" code, I had considerably less skill
in programming at that time.  In fact, work on Services accounted for a
significant amount of my programming experience during its first several
years of existence, and careful reading of the Services source code may
well reveal the effect of that experience.</p>

<p>My private IRC server eventually expanded into a small network, as
friends of mine connected servers of their own.  I stored a copy of the
Services source code on each server, both to function as a backup of the
code itself and to allow a backup copy of Services to be started in case
the primary one should fail for some reason.  This backfired, however, when
one such server was broken into on November 19, 1996; a few days later, we
received a message essentially accusing us of stealing Services from
another IRC network called IceNet.  Investigation showed the purported
"original" Services to in fact be a copy of my own program, presumably
taken from the broken-into server.  I was naturally upset, and exchanged
several angry messages with an IceNet administrator, but the situation was
amicably resolved in the end, and IceNet continued using my program.</p>

<p>After this incident, I began receiving occasional requests for copies of
the program, and at some point over the next month or two I decided that,
since so many people seemed interested in it and I had no particular reason
to keep it secret, I would release the source code publicly, free for
anyone to use.  The first public release was version 2.2.0, on January 10,
1997.  This sudden "change of heart" is also the reason Services has such a
generic name: I had not originally planned to publish the program, so I had
not bothered to come up with a distinctive name for it.  The moniker
"EsperNet Services", from the name of the network for which I developed it,
appeared every now and then, but (partly to avoid users of the program
flooding EsperNet administrators with help requests) I only called it
"Services".  The current full name, "Services for IRC Networks", was chosen
when the program was listed on the <a href="http://freshmeat.net/">FreshMeat
software index</a> <span class="remotehost">[freshmeat.net]</span>.</p>

<p>I continued Services development through mid-1999, extending the program
to deal with users' feature requests and my own experience.  But after
graduating from university and moving to Japan to work, I found myself with
little time available for work on Services; so, after doing some
preliminary work for version 4.3, I handed the code base off to Andrew
Kempe, a user of Services who had contributed numerous suggestions and
improvements.  At the time, I had already had some preliminary thoughts
toward redesigning Services, including the module system which eventually
saw the light of day in version 5.0, but as my IRC use had declined
significantly, I had intended to just let Services go, to be developed by
others with more interest.</p>

<p>However, in late 2000, Andrew Kempe wrote to me explaining that he no
longer had time to work on Services, and I ended up deciding to continue
Services development where he had left off.  I reviewed his additions,
notably the session limiting system (now the <tt>operserv/sessions</tt>
module), and began preparations for releasing version 4.5.  I also began
using the CVS source code management tool to keep better track of changes
to the code.  (I had at one point considered taking advantage of CVS to
allow others to update the code as well, but copyright issues aside, there
were simply very few people I trusted to write code to my standards, and
with the exception for a time of a friend from EsperNet, I was the only one
who actually developed the code.)</p>

<p>After releasing version 4.5.0, I decided to start work on a major
redesign of Services, including the implementation of my former module
idea.  Over the five years until this time, the Services source code had
not undergone any significant redesign, and the result of continually
adding features on top of other features was, to put it politely, a mess.
In one of my favorite computer science books, <i>The Mythical Man-Month</i>,
the author, Dr. Frederick Brooks, gives as one of the tenets of software
development: "<i>Plan to throw one away; you will, anyhow</i>" (chapter
11).  Indeed, I had learned a lot about both software design and
programming in general and IRC service software in particular over the
years, and it was easy to see where the current Services code was
inadequate or poorly designed.  So, around the release of version 4.5.8, I
split off a copy of the code base, added in the module support system I had
started work on separately, and began a year-plus effort to clean up the
code.  I did not actually rewrite the program from scratch, since I lacked
both the time and the incentive to do so, but the structure of the program
was vastly improved.</p>

<p>The one major goal for version 5.0 which I did not complete was a new
database implementation.  The database system which existed at the time was
heavily intertwined with the individual pseudoclients (NickServ, ChanServ,
and so on), making it very difficult for third parties to add new features
that required persistent data; this was exacerbated by the database file
format, originally just a binary dump of the data structures stored in
memory, which lacked any sort of flexibility at all and was highly
susceptible to data corruption.  Every time a new field was added, the
database files' identifying version number was incremented, rendering the
files incompatible with previous versions, and requiring highly complex
routines in later releases to handle data from earlier versions.  It had
been my intention to implement a format-agnostic system for 5.0 along with
a more expandable and robust file format, but this proved to be too much to
handle at once, and I ended up simply moving all of the database handling
to a module (<tt>database/version4</tt>) that used dozens of hacks to read
and write the data in a format that remained compatible with the previous
version (4.5).</p>

<p>After a couple of years of on-and-off work, I finally found the time to
go back and redesign the database system, and the result is the code
present in this version.  However, as I no longer use IRC, my interest in
continuing development of Services has waned, and I have ended Services
development as of version 5.1.</p>

<p class="backlink"><a href="#top">Back to top</a></p>

<!------------------------------------------------------------------------>
<hr/>

<p class="backlink"><a href="index.html">Table of Contents</a> |
<a href="2.html">Next section: Core Services functionality</a></p>

</body>
</html>
Revision:	3389
Committed:	Fri Apr 25 14:12:15 2014 UTC (12 years, 2 months ago) by michael
Content type:	text/html
File size:	12677 byte(s)
Log Message:	- Imported ircservices-5.1.24
#	User	Rev	Content
1	michael	3389	<?xml version="1.0" encoding="ISO-8859-1"?>
2			<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11-strict.dtd">
3			<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
4			<head>
5			<meta http-equiv="Content-Style-Type" content="text/css"/>
6			<style type="text/css">@import "style.css";</style>
7			<title>IRC Services Technical Reference Manual - 1. Program structure overview</title>
8			</head>
9
10			<body>
11			<h1 class="title" id="top">IRC Services Technical Reference Manual</h1>
12
13			<h2 class="section-title">1. About this manual</h2>
14
15			<p class="section-toc">
16			1-1. <a href="#s1">Intended audience</a>
17			<br/>1-2. <a href="#s2">Structure of this manual</a>
18			<br/>1-3. <a href="#s3">History of Services development</a>
19			</p>
20
21			<p class="backlink"><a href="index.html">Table of Contents</a> \|
22			<a href="2.html">Next section: Core Services functionality</a></p>
23
24			<!------------------------------------------------------------------------>
25			<hr/>
26
27			<h3 class="subsection-title" id="s1">1-1. Intended audience</h3>
28
29			<p>This manual is written with an intended audience of programmers looking
30			to change or extend the IRC Services source code, and covers the overall
31			structure of the code as well as details on each of Services' functions.
32			The content of this manual is highly technical, and requires a reasonably
33			high level of expertise (both in programming in general and in the C
34			language and Unix computer systems in particular) to peruse.</p>
35
36			<p>For information on installing and using IRC Services, please see the
37			<a href="../index.html">user's manual</a>.</p>
38
39			<p>Note that this manual describes Services for IRC Networks version 5.1.24,
40			and its contents may not be applicable to modified versions or derivative
41			programs. Also, this manual is intended to be used alongside the source
42			code; details of function parameters, structure fields, and the like are
43			often omitted for the sake of brevity and clarity. <!-- Use the source,
44			Luke! --> </p>
45
46
47			<p class="backlink"><a href="#top">Back to top</a></p>
48
49			<!------------------------------------------------------------------------>
50			<hr/>
51
52			<h3 class="subsection-title" id="s2">1-2. Structure of this manual</h3>
53
54			<p>Services can be broken up into two parts: the <b>core functionality</b>,
55			which implements the "IRC server" part of Services, and <b>modules</b>,
56			which add the "bot-like" functions to Services as well as provide common
57			interfaces to differing implementations of some functions.</p>
58
59			<p>The core Services functionality is primarily discussed in
60			<a href="2.html">section 2</a>. Two topics, however, are given their own
61			sections due to their complexity: communications (<a href="3.html">section
62			3</a>) and module handling (<a href="4.html">section 4</a>).</p>
63
64			<p>Modules are divided into four groups:</p>
65
66			<ul>
67			<li><b>protocol modules</b> (<a href="5.html">section 5</a>), which
68			handle the interface between the generic IRC server implemented by the
69			Services core and the particular protocols used by various IRC servers;</li>
70			<li><b>database modules</b> (<a href="6.html">section 6</a>), which handle
71			the storage and retrieval of data used by pseudoclients and other Services
72			functions;</li>
73			<li><b>pseudoclient modules</b> (<a href="7.html">section 7</a>), which
74			implement the various pseudoclients themselves; and</li>
75			<li><b>other modules</b> (<a href="8.html">section 8</a>), including all
76			modules not covered above.</li>
77			</ul>
78
79			<p><a href="9.html">Section 9</a> covers <tt>convert-db</tt>, the tool
80			included with Services to convert databases from other Services-like
81			programs to an XML data stream that can then be imported into Services
82			itself. The compilation of Services (the <tt>configure</tt> script and
83			<tt>Makefile</tt> files) is described in <a href="10.html">section 10</a>,
84			and areas where Services could potentially be improved are discussed in
85			<a href="11.html">section 11</a>.</p>
86
87			<p>A source code reference is included as <a href="a.html">Appendix A</a>
88			(by source file) and <a href="b.html">Appendix B</a> (by function name).</p>
89
90			<p class="backlink"><a href="#top">Back to top</a></p>
91
92			<!------------------------------------------------------------------------>
93			<hr/>
94
95			<h3 class="subsection-title" id="s3">1-3. History of Services development</h3>
96
97			<p>Before discussing the details of the Services code base, it is worth
98			mentioning the history of Services development.</p>
99
100			<p>Originally, I (Andrew Church, the developer of Services) started writing
101			Services as something of a "quick hack" for my personal server, which I had
102			set up for the use of a channel I frequented at the time that had decided
103			to move away from the DALnet IRC network. DALnet did not at the time—and
104			to the best of my knowledge still does not—publish the source code to
105			their nickname and channel registration services, nor was I able to locate
106			any other similar usable open-source software, so I took the route of
107			writing my own. This motive had, at least initially, significant influence
108			on the user interface design; the service nicknames NickServ and ChanServ
109			seem to have been around since the early days of IRC, but the basic
110			commands, such as <tt>REGISTER</tt>, <tt>SET</tt>, and the NickServ
111			<tt>RECOVER</tt> and <tt>RELEASE</tt> functions, were based on those seen
112			on DALnet. (Ironically, it seems that DALnet has since incorporated at
113			least one feature, the channel successor concept, from this program due to
114			user pressure.)</p>
115
116			<p>The first "release" of Services—using the term loosely,
117			since I had no interest in releasing Services publicly at this
118			point—was made on April 5, 1996, when the program first went online.
119			Since my IRC server was mainly for private use, there was little call for
120			additional features, and aside from occasional bug fixes, not much
121			development was performed during this time. The code base itself could be
122			called "functional" at best; while I have always put effort into writing
123			good code rather than simply "working" code, I had considerably less skill
124			in programming at that time. In fact, work on Services accounted for a
125			significant amount of my programming experience during its first several
126			years of existence, and careful reading of the Services source code may
127			well reveal the effect of that experience.</p>
128
129			<p>My private IRC server eventually expanded into a small network, as
130			friends of mine connected servers of their own. I stored a copy of the
131			Services source code on each server, both to function as a backup of the
132			code itself and to allow a backup copy of Services to be started in case
133			the primary one should fail for some reason. This backfired, however, when
134			one such server was broken into on November 19, 1996; a few days later, we
135			received a message essentially accusing us of stealing Services from
136			another IRC network called IceNet. Investigation showed the purported
137			"original" Services to in fact be a copy of my own program, presumably
138			taken from the broken-into server. I was naturally upset, and exchanged
139			several angry messages with an IceNet administrator, but the situation was
140			amicably resolved in the end, and IceNet continued using my program.</p>
141
142			<p>After this incident, I began receiving occasional requests for copies of
143			the program, and at some point over the next month or two I decided that,
144			since so many people seemed interested in it and I had no particular reason
145			to keep it secret, I would release the source code publicly, free for
146			anyone to use. The first public release was version 2.2.0, on January 10,
147			1997. This sudden "change of heart" is also the reason Services has such a
148			generic name: I had not originally planned to publish the program, so I had
149			not bothered to come up with a distinctive name for it. The moniker
150			"EsperNet Services", from the name of the network for which I developed it,
151			appeared every now and then, but (partly to avoid users of the program
152			flooding EsperNet administrators with help requests) I only called it
153			"Services". The current full name, "Services for IRC Networks", was chosen
154			when the program was listed on the <a href="http://freshmeat.net/">FreshMeat
155			software index</a> <span class="remotehost">[freshmeat.net]</span>.</p>
156
157			<p>I continued Services development through mid-1999, extending the program
158			to deal with users' feature requests and my own experience. But after
159			graduating from university and moving to Japan to work, I found myself with
160			little time available for work on Services; so, after doing some
161			preliminary work for version 4.3, I handed the code base off to Andrew
162			Kempe, a user of Services who had contributed numerous suggestions and
163			improvements. At the time, I had already had some preliminary thoughts
164			toward redesigning Services, including the module system which eventually
165			saw the light of day in version 5.0, but as my IRC use had declined
166			significantly, I had intended to just let Services go, to be developed by
167			others with more interest.</p>
168
169			<p>However, in late 2000, Andrew Kempe wrote to me explaining that he no
170			longer had time to work on Services, and I ended up deciding to continue
171			Services development where he had left off. I reviewed his additions,
172			notably the session limiting system (now the <tt>operserv/sessions</tt>
173			module), and began preparations for releasing version 4.5. I also began
174			using the CVS source code management tool to keep better track of changes
175			to the code. (I had at one point considered taking advantage of CVS to
176			allow others to update the code as well, but copyright issues aside, there
177			were simply very few people I trusted to write code to my standards, and
178			with the exception for a time of a friend from EsperNet, I was the only one
179			who actually developed the code.)</p>
180
181			<p>After releasing version 4.5.0, I decided to start work on a major
182			redesign of Services, including the implementation of my former module
183			idea. Over the five years until this time, the Services source code had
184			not undergone any significant redesign, and the result of continually
185			adding features on top of other features was, to put it politely, a mess.
186			In one of my favorite computer science books, <i>The Mythical Man-Month</i>,
187			the author, Dr. Frederick Brooks, gives as one of the tenets of software
188			development: "<i>Plan to throw one away; you will, anyhow</i>" (chapter
189			11). Indeed, I had learned a lot about both software design and
190			programming in general and IRC service software in particular over the
191			years, and it was easy to see where the current Services code was
192			inadequate or poorly designed. So, around the release of version 4.5.8, I
193			split off a copy of the code base, added in the module support system I had
194			started work on separately, and began a year-plus effort to clean up the
195			code. I did not actually rewrite the program from scratch, since I lacked
196			both the time and the incentive to do so, but the structure of the program
197			was vastly improved.</p>
198
199			<p>The one major goal for version 5.0 which I did not complete was a new
200			database implementation. The database system which existed at the time was
201			heavily intertwined with the individual pseudoclients (NickServ, ChanServ,
202			and so on), making it very difficult for third parties to add new features
203			that required persistent data; this was exacerbated by the database file
204			format, originally just a binary dump of the data structures stored in
205			memory, which lacked any sort of flexibility at all and was highly
206			susceptible to data corruption. Every time a new field was added, the
207			database files' identifying version number was incremented, rendering the
208			files incompatible with previous versions, and requiring highly complex
209			routines in later releases to handle data from earlier versions. It had
210			been my intention to implement a format-agnostic system for 5.0 along with
211			a more expandable and robust file format, but this proved to be too much to
212			handle at once, and I ended up simply moving all of the database handling
213			to a module (<tt>database/version4</tt>) that used dozens of hacks to read
214			and write the data in a format that remained compatible with the previous
215			version (4.5).</p>
216
217			<p>After a couple of years of on-and-off work, I finally found the time to
218			go back and redesign the database system, and the result is the code
219			present in this version. However, as I no longer use IRC, my interest in
220			continuing development of Services has waned, and I have ended Services
221			development as of version 5.1.</p>
222
223			<p class="backlink"><a href="#top">Back to top</a></p>
224
225			<!------------------------------------------------------------------------>
226			<hr/>
227
228			<p class="backlink"><a href="index.html">Table of Contents</a> \|
229			<a href="2.html">Next section: Core Services functionality</a></p>
230
231			</body>
232			</html>