riff_site/cache/pages/internals.html

<p class="lead">When trying to guide someone into adding a feature in Nikola, it hit me that
while the way it's structured makes sense <strong>to me</strong> it is far from obvious.</p>
<p>So, this is a short document explaining what each piece of Nikola does and
how it all fits together.</p>
<dl>
<dt>Nikola is a Pile of Plugins</dt>
<dd><p>Most of Nikola is implemented as plugins using <a class="reference external" href="http://yapsy.sourceforge.net/">Yapsy</a>.
You can ignore that they are plugins and just think of them as regular python
modules and packages with a funny little <code class="docutils literal">.plugin</code> file next to them.</p>
<p>So, 90% of the time, what you want to do is either write a new plugin or extend
an existing one.</p>
<p>There are several kinds of plugins, all implementing interfaces defined in
<code class="docutils literal">nikola/plugin_categories.py</code> and documented in
<a class="reference external" href="https://getnikola.com/extending.html">Extending Nikola</a></p>
<p>If your plugin has a dependency, please make sure it doesn't make Nikola
throw an exception when the dependency is missing. Try to fail gracefully
with an informative message.</p>
</dd>
<dt>Commands are plugins</dt>
<dd><p>When you use <code class="docutils literal">nikola foo</code> you are using the plugin <code class="docutils literal">command/foo</code>. Those are
used to extend Nikola's command line. Their interface is defined in the <code class="docutils literal">Command</code>
class. They take options and arguments and do whatever you want, so go wild.</p>
</dd>
<dt>The <code class="docutils literal">build</code> command is special</dt>
<dd><p>The <code class="docutils literal">build</code> command triggers a whole lot of things, and is the core of Nikola
because it's the one that you use to build sites. So it deserves its own section.</p>
</dd>
</dl>
<section id="the-build-command">
<h1>The Build Command</h1>
<p>Nikola's goal is similar, deep at heart, to a Makefile. Take sources, compile them
into something, in this case a website. Instead of a Makefile, Nikola uses
<a class="reference external" href="https://pydoit.org">doit</a>.</p>
<p>Doit has the concept of &quot;tasks&quot;. The 1 minute summary of tasks is that they have:</p>
<dl class="simple">
<dt>actions</dt>
<dd><p>What the task <strong>does</strong>. For example, convert a markdown document into HTML.</p>
</dd>
<dt>dependencies</dt>
<dd><p>If this file changes, then we need to redo the actions. If this configuration
option changes, redo it, etc.</p>
</dd>
<dt>targets</dt>
<dd><p>Files that the action generates. No two actions can have the same targets.</p>
</dd>
<dt>basename:name</dt>
<dd><p>Each task is identified by either a name or a basename:name pair.</p>
</dd>
</dl>
<aside class="sidebar">
<p class="sidebar-title">More about tasks</p>
<p>If you ever want to do your own tasks, you really should read the doit
<a class="reference external" href="https://pydoit.org/tasks.html">documentation on tasks</a>.</p>
<p>Notably, by default doit redirects <code class="docutils literal">stdout</code> and <code class="docutils literal">stderr</code>. To get a
proper PDB debugging shell, you need to use doit's own
<a class="reference external" href="https://pydoit.org/tools.html#set-trace">set_trace</a> function.</p>
</aside>
<p>So, what Nikola does, when you use the build command, is to read the
configuration <code class="docutils literal">conf.py</code> from the current folder, instantiate
the <code class="docutils literal">Nikola</code> class, and have it generate a whole list of tasks for doit
to process. Then doit will decide which tasks need doing, and do them, in
the right order.</p>
<p>The place where the tasks are generated is in <code class="docutils literal">Nikola.gen_tasks</code>, which collects tasks
from all the plugins inheriting <code class="docutils literal">BaseTask</code>, massages them a bit, then passes them
to doit.</p>
<p>So, if you want things to happen on <code class="docutils literal">build</code> you want to create a Task plugin, or extend
one of the existing ones.</p>
<aside class="sidebar">
<p class="sidebar-title">Tests</p>
<p>While Nikola is not a hardcore TDD project, we like tests. So, please add them if you can.
You can write unit tests or integration tests. (Doctests are not supported
anymore due to fragility.)</p>
</aside>
</section>
<section id="posts-and-pages">
<h1>Posts and Pages</h1>
<p>Nikola has a concept of posts and pages. Both are more or less the same thing, except
posts are added into RSS feeds and pages are not. All of them are in a list called
&quot;the timeline&quot; formed by objects of class <code class="docutils literal">Post</code>.</p>
<p>When you are creating a task that needs the list of posts and/or pages (for example,
the RSS creation plugin) on task execution time, your plugin should call <code class="docutils literal">self.site.scan_posts()</code>
in <code class="docutils literal">gen_tasks</code> to ensure the timeline is created and available in
<code class="docutils literal">self.site.timeline</code>. You should not modify the timeline, because it will cause consistency issues.</p>
<aside class="sidebar">
<p class="sidebar-title">scan_posts</p>
<p>The <code class="docutils literal">Nikola.scan_posts</code> function can be used in plugins to force the
timeline creation, for example, while creating the tasks.</p>
</aside>
<p>Your plugin can use the timeline to generate &quot;stuff&quot; (technical term). For example,
Nikola comes with plugins that use the timeline to create a website (surprised?).</p>
<p>The workflow included with nikola is as follows (incomplete!):</p>
<ol class="arabic simple">
<li><p>The post is assigned a compiler based on its extension and the <code class="docutils literal">COMPILERS</code> option.</p></li>
<li><p>The compiler is applied to the post data and a &quot;HTML fragment&quot; is produced. That
fragment is stored in a cache (the <code class="docutils literal">posts</code> plugin).</p></li>
<li><p>The configured theme has templates (and a template engine), which are applied to the post's
HTML fragment and metadata (the <code class="docutils literal">pages</code> plugin).</p></li>
<li><p>The original sources for the post are copied to some accessible place (the <code class="docutils literal">sources</code> plugin).</p></li>
<li><p>If the post is tagged, some pages and RSS feeds for each tag are updated (the <code class="docutils literal">tags</code> plugin).</p></li>
<li><p>If the post is new, it's included in the blog's RSS feed (the <code class="docutils literal">rss</code> plugin).</p></li>
<li><p>The post is added in the right place in the index pages for the blog (the <code class="docutils literal">indexes</code> plugin).</p></li>
<li><p>CSS/JS/Images for the theme are put in the right places (the <code class="docutils literal">copy_assets</code> and <code class="docutils literal">bundles</code> plugins).</p></li>
<li><p>A File describing the whole site is created (the <code class="docutils literal">sitemap</code> plugin).</p></li>
</ol>
<p>You can add whatever you want to that list: just create a plugin for it.</p>
<p>You can also expand Nikola's capabilities at several points:</p>
<dl class="simple">
<dt>compilers</dt>
<dd><p>Nikola supports a variety of markups. If you want to add another one, you need to create
a <code class="docutils literal">Compiler</code> plugin.</p>
</dd>
<dt>templates</dt>
<dd><p>Nikola's themes can use Jinja2 or Mako templates. If you prefer another template system,
you have to create a <code class="docutils literal">TemplateSystem</code> plugin.</p>
</dd>
<dt>themes</dt>
<dd><p>To change how the generated site looks, you can create custom themes.</p>
</dd>
</dl>
<p>And of course, you can also replace or extend each of the existing plugins.</p>
<section id="nikola-architecture">
<h2>Nikola Architecture</h2>
<a class="reference external image-reference" href="https://getnikola.com/images/architecture.png"><img alt="https://getnikola.com/images/architecture.thumbnail.png" src="https://getnikola.com/images/architecture.thumbnail.png" /></a>
</section>
</section>