Taeluf.com

Code Scrawl

Code Scrawl is a documentation generator. You write .src.md files and call buit-in functions with @‌verbs() within, including @‌template(template_name) to load several built-in templates. You can create custom extensions and templates for your own projects, or to generate documentation for using a library you made.

Also generate full API documentation of PHP classes. AST Generation uses taeluf/lexer, which can be extended to support other languages, but as of May 2023, there are no clear plans to do this.

Features

  • Integrated AST generation to automatically copy+paste specific parts of source code, thanks to php/lexer
  • Several builtin verbs and templates
  • Extend with custom verbs and templates.

Deprecation Warning:

  • v1.0 will stop using hidden directories like .docsrc/ and instead use docsrc/. The defaults.json file will be changed to accomodate in v1.0
  • v1.0 will use dir src/ instead of code/ as a default directory to scan. (currently also scans test/, which will not change)

Install

Choose one:

  • composer require taeluf/code-scrawl v0.8.x-dev
  • composer require taeluf/code-scrawl {latest_version} - see Versions
  • PHAR (experimental): Set the version, then curl to download the phar file.
    1. v="0.X.Y.Z" - See Versions
    2. curl -o scrawl.phar https://gitlab.com/taeluf/php/CodeScrawl/-/raw/$v/bin/scrawl.phar?inline=false

Version Numbers: 0.X.Y.Z, 0.X is from the branch name, Y is bumped with feature additions, Z is bumped with bugfixes and some other changes. (This is determined by the commit messages prefix of 'feature:', 'bugfix:', or 'other:'. And if Y is bumped, Z is set to 0.)

Configure

Create a file .config/scrawl.json or scrawl.json in your project root. Defaults:

{  
    "--NOTICE":"v1.0 will introduce updated defaults.",  
  
    "template.dirs": [".doctemplate"],  
  
    "dir.docs": "docs",  
    "dir.src": ".docsrc",  
    "dir.scan": ["code", "test"],  
    "ScrawlExtensions":[],  
  
    "file.bootstrap":"scrawl-bootstrap.php",  
  
    "api.output_dir": "api/",  
    "api.generate_readme": true,  
  
    "deleteExistingDocs": false,  
    "readme.copyFromDocs": false,  
  
    "markdown.preserveNewLines": true,  
    "markdown.prependGenNotice": true  
}

Default Configs / File Structre

Note: Instructions updated to use non-hidden directories, but the default.json file still uses hidden directories. This inconsistency will be fixed in v1.0

  • config/scrawl.json: configuration file
  • docsrc/*.src.md: documentation source files (from which your documentation is generated)
  • docs/: generated documentation output
  • src/*, and test/*: Code files to scan
  • doctemplate/*.md.php and CodeScrawl/src/Template/*.md.php: re-usable templates
  • scrawl-bootstrap.php: Runs at start of $scrawl->run() and $this is the $scrawl instance.

Usage

Overview:

  • Execute with vendor/bin/scrawl from your project root.
  • Write files like docsrc/README.src.md
  • Use Markdown Verbs (mdverb) to load documentation and code into your .src.md files: @‌file(src/defaults.json) would print the content of src/defaults.json
  • Use the @‌template mdverb to load a template: @‌template(php/compose_install, taeluf/code-scrawl) to print composer install instructions
  • Use the @‌ast mdverb to load code from the AST (Asymmetric Syntax Tree): @‌ast(class.\Tlf\Scrawl.docblock.description) to print directly or @‌ast(class.\Tlf\Scrawl, ast/class) to use an ast template.
  • Write special @‌codeverbs in comments in code source files to export docblocks and code.
  • Extend it! Write custom templates and @‌mdverb handlers
  • Write custom md verb handlers (see 'Extension' section below)
  • Write custom templates (see 'Extension' section below)
  • Use an ASCII Non-Joiner character after an @ sign, to write a literal @‌at_sign_with_text and not execute the verb handler.

Write Documents: Example

Write files in your docsrc folder with the extension .src.md.
Example, from docsrc/README.src.md

This would display the ## Install instructions and ## Configure instructions as above

# Code Scrawl  
Intro text  
  
## Setup   
### Install  
@‌template(php/compose_install, taeluf/code-scrawl)  
  
### Configure  
Create a file `.config/scrawl.json` or `scrawl.json` in your project root. Defaults:  
‌```json  
@‌file(src/defaults.json)  
‌```

@‌MdVerbs List

Write these in your markdown source files for special functionality

  • @import(): Import something previously exported with @export or @export_start/@export_end. Usage: @import(Namespace.Key)
  • @file(): Copy a file's content into your markdown.. Usage: @file(rel/path/to/file.ext)
  • @template(): Load a template. Usage: @template(template_name, arg1, arg2)
  • @link(): Output links configured in your config json file.
    Config format is {..., "links": { "link_name": "https://example.org"} }. Usage: @link(phpunit)
  • @easy_link(): Get a link to common services (twitter, gitlab, github, facebook). Usage: @easy_link(twitter, TaelufDev)
  • @hard_link(): just returns a regular markdown link. In future, may check validity of link or do some kind of logging. Usage: @hard_link(https://url.com, LinkName)
  • @see_file(): Get a link to a file in your repo. Usage: @see_file(relative/file/path)
  • @see(): Get a link to a file in your repo. Usage: @see_file(relative/file/path)
  • @system(): Run a command on the computer's operating system.

Supported options: trim, trim_empty_lines, last_x_lines, int

  • @ast(): Get an ast & optionally load a custom template for it. Usage: @ast(class.ClassName.methods.docblock.description, ast/default)

Template List

Templates can be loaded with @‌template(template_name, arg1, arg2), except ast/* templates (see below for those).

Each template contains documentation for how to use it & what args it requres.

@‌ast() Templates (Asymmetric Syntax Tree)

Example: @‌ast(class.\Tlf\Scrawl.docblock.description) to print directly or @‌ast(class.\Tlf\Scrawl, ast/class) to use an ast template.

@‌CodeVerbs Exports (from code in scan dirs)

In a docblock, write @‌export(Some.Key) to export everything above it.

In a block of code (or literally anywhere), write // @‌export_start(Some.Key) then // @‌export_end(Some.Key) to export everything in between.

See test/run/SrcCode.php for examples

Extensions

Templates: Write a custom template

Look at existing templates in doctemplate/ or src/Template for examples.

Verb Handlers: Write a custom verb handler

In your scrawl-bootstrap.php file, do something like:

<?php  
/**  
 * @param $scrawl instance of `\Tlf\Scrawl`  
 */  
  
$scrawl->verb_handlers['own_verb'] =   
    function($arg1, $arg2){  
        return $arg1.'--'.$arg2;  
  
    };

PHP Extensions

Create a class that implements \Tlf\Scrawl\Extension. See file src/Extension.php. You can class YourExt extends \Tlf\Scrawl\DoNothingExtension as a base class and then only override the methods you need.

In your config.json, set "ScrawlExtensions": ["Your\\Fully\\Qualified\\ClassName"], with as many extension classes as you like.

Architecture

  • Tlf\Scrawl: Primary class that run()s everything.
  • \Tlf\Scrawl->run(): generate api/* files. Load all php classes. scan code files. Scan .src.md files & output .md files.
    • file set as file.bootstrap is require()d at start of run() to setup additional mdverb handlers and (eventually) other extensions. Default is scrawl-bootstrap.php
  • Tlf\Scrawl\Ext\MdVerbs: Instantiated during run() prior to scanning documentation source files. When an @‌mdverb(arg1,arg2) is encounter in documentation source file, a handler ($mdverbs->handlers['mdverb']) is called with string arguments like $handler('arg1', 'arg2'). Extend it by adding a callable to $scrawl->mdverb_ext->handlers.
    • Tlf\Scrawl\Ext\MdVerb\MainVerbs: Has functions for simple verb handlers like @‌file, @‌template. Adds each function as a handler to the MdVerbs class.
    • \Tlf\Scrawl\Ext\MdVerb\Ast: A special mdverb handler that loads ASTs from the lexer and passes it to a named (or default) template.
  • Tlf\Scrawl\FileExt\Php: The only AST extension currently active. It is a convenience class that wraps the Lexer so it is easily called by Scrawl. It is called to setup ASTs by class.ClassName... on $scrawl. Call $scrawl->get('ast', 'class.ClassName...'). It is called to generate the api/* documentation files.
  • INACTIVE CLASS Tlf\Scrawl\Ext\Main simply copies the project_root/.docrsc/README.src.md to project_root/README.md
  • Tlf\Scrawl\FileExt\ExportDocBlock and ExportStartEnd handle the @‌export() tag in docblocks and the @‌export_start()/@‌export_end() tags.
  • \Tlf\Scrawl\Utility\DocBlock: Convenience class for extracting docblocks from files.
  • \Tlf\Scrawl\Utility\Main: Class with some convenience functions
  • \Tlf\Scrawl\Utility\Regex: Class to make regex matching within a file more abstract & object oriented. (it's not particularly good)

More Info

  • Run withOUT the cli: Some of the configs require absolute paths when running through php, rather than from the cli. An example is in test/run/Integrate.php under method testRunFull()
  • @‌literal: Displaying a literal @‌literal in an md source file can be done by putting a Zero Width Non-Joiner after the arroba (@). You can also use a backslash like so: @\literal, but then the backslash prints
  • All classes in this repo: docs/ClassList.md
  • $scrawl has a get/set feature. In a template, you can get an ast like $scrawl->get('ast.Fully\Qualified\ClassName'). Outside a template, you can use the Php class (src/Ext/Php.php) to get an ast from a file or string.
  • the deleteExistingDocs config has a bit of a sanity check to make sure you don't accidentally delete your whole system or something ...