====== Source Plugin ====== ---- plugin ---- description: Allows you to include all or part of the contents of another file, with syntax highlighting, into the current page author : Christopher Smith email : chris@jalakai.co.uk type : syntax lastupdate : 2008-08-13 compatible : Hrun depends : conflicts : similar : tags : code, syntaxhighlight, include, file downloadurl: http://dokuwiki.jalakai.co.uk/plugin-source.tar.gz ---- ===== Syntax ===== * **//filename//** --- required, the name of the file to be included. This can either be a normal path & file name or a URI. If you wish to use file URI's then ''allow_url_fopen=On'' must be set in the webserver's php.ini file (also refer to the [[#security|location]] setting and the [[#warnings|security warning]] below). If //filename// includes spaces it must be surrounded by a matched pair of quotes (''%%'%%'' or ''%%"%%''). * **//#start-end//** --- optional, only show lines from ''start'' to ''end''. First line in the file is counted as 1. Both ''start'' and ''end'' must be positive numbers. If ''end'' is less than ''start'' or ''start'' is greater than there are lines in the file nothing will be shown. * **//language//** --- optional, the language string to be passed to [[http://qbnz.com/highlighter/|GeSHi]] for syntax highlighting. This plug-in accepts the same language strings as [[:wiki:syntax#syntax_highlighting|DokuWiki's markup]]. If the language string is not given, the plugin will attempt to deduce the language from the file extension. * **//title//** --- optional, a title to be displayed above the file contents. Everything after the pipe (''|'') is treated as the ''title''. If not present the file name will be used, e.g. //"file: ''filename''"// You can see the plugin in action [[http://wiki.jalakai.co.uk/dokuwiki/doku.php/wiki/plugin/source_demonstration|here]]. ===== Warnings ===== Indiscriminate use of this plug in can be a serious security risk to your web server. It has the potential to expose any file on the web server computer that is accessible to the web server software. In particular: * ''[[http://www.php.net/manual/en/ref.filesystem.php#ini.allow-url-fopen|allow_url_fopen=On]]'' is a potential security risk. If you use this setting, I strongly recommend ensuring 'php' is included in the denied extension list (refer ''[[#security|$deny]]'' setting below). * ensure the php.ini setting ''[[http://www.php.net/manual/en/features.safe-mode.php#ini.open-basedir|open_basedir]]'' is correctly set. If not, this plugin has the potential to expose any file accessible to the webserver to the page creator/editor. * Attempting to use URIs to link in PHP source files resident on a PHP enabled webserver will result in the webserver attempting to process the PHP script. This can have unpredictable results. * Be aware that often a dynamic website will have files containing plain text passwords for access to other server systems (e.g. databases). \\ For further information, see [[http://www.php.net/manual/en/security.filesystem.php|PHP File System Security]] ===== Settings ===== ==== Security ==== These settings can be altered using the configuration page accessed via the admin menu. * **//location//** --- This value will be preprended to all file names. It can be used to restrict the portion of the file space (or URI space) exposed to the ''%%%%'' command. \\ \\ By setting ''$location=%%'http://'%%'' you can force all file retrieval through webservers and thereby make use of the webservers' own restrictions (e.g. in Apache systems, DocumentRoot, .htaccess files) which are likely to be stronger than that of the file system itself. \\ \\ If there is a value for ''$location'' all file names which include the path traversal ".." will be rejected. \\ \\ * **//allow//** --- An array of file extension strings. If this array is empty it is ignored. If it has any members only files with the listed extensions may be "sourced". * **//deny//** --- An array of file extension strings. If ''$allow'' is empty, no file with an extension included in ''$deny'' may be "sourced". If ''$allow'' is non-empty ''$deny'' is ignored. * **//rules//** --- Rules allow far more complex file inclusion restrictions to be constructed than with the ''location'', ''allow'' & ''deny'' settings. * one rule per line. * each rule line must consist of 'allow' or 'deny' followed by a valid php regular expression, including the delimeters. * comment lines can be included by making the first character on the line a semi-colon, ';'. * blank lines can be included and are ignored * e.g. ; deny everything then allow HTML files DENY /./ ALLOW /\.htm(l)?/ ==== Other ==== This setting is included in the plugin's ''syntax.php'' file. * **//$extensions//** --- An associative array matching file extensions to recognised GeSHi languages. Value pairs are of the form ''"file-extension" %%=>%% "GeSHI-language"''. ===== Installation ===== Search and install the plugin using the [[plugin:extension|Extension Manager]]. Refer to [[:Plugins]] on how to install plugins manually. * [[http://dokuwiki.jalakai.co.uk/plugin-source.zip|zip format (8k)]] * [[http://dokuwiki.jalakai.co.uk/plugin-source.tar.gz|tar.gz format (5k)]] ===== Revision History ===== * 2008-08-13 --- update plugin URL * 2006-12-24 --- major update * use localisation for strings * add action plugin component to work with DokuWiki caching. \\ where source file modification time is available, the plugin will use it to assist in determination of page cache validity. If the information is not available the cache will always be expired. * add support for (basic) rules to govern source file access. The rules work in addition to the earlier 'location','allow' & 'deny' criteria. It is probably sensible to use one or the other. That is, if you use rules set the other three settings to empty string. * add support for line number selection. * add support for any renderer format. If a format other than 'XHTML' or 'metadata' is chosen, the plugin reads the appropriate lines from the source file and passes them to DokuWiki's ->code() method to be rendered in the same was as a code block. * 2006-05-01 --- (darcs version only, others to follow) Settings updated to use [[devel:common plugin functions]] which will allow them to be edited using the [[plugin:config|Configuration Manger]] in the admin menu. * 2005-08-23 --- updated; supports filenames with spaces. * 2005-08-19 --- updated; ".." directory traversal vulnerability removed, user specifiable title added, downloadable plugin package released. * 2005-07-12 --- released. ===== To Do ===== * Improve checks on file name to prevent use of ".." --- DONE * More advanced rules ===== Discussion ===== This allows you to include external pages with no **
** formatting.  We only use this with loading internal web-server pages in a simple proxy fashion.  Because of this we further locked down the configuration in syntax.php.  When the configuration is set like this, you must not enter the %%http://%% prefix, since it's applied for you.  Also, it must end in HTML.  There may be some XSS attacks here... but our wiki is only available to our users.   --- //[[caylan@aero.und.edu|Caylan Larson]] 2005-10-06 16:48//

  // syntax.php configuration at line 36
  $location = 'http://';
    
  // if $allow array contains any elements, ONLY files with the extensions listed will be allowed
  $allow = array('html');
  
  // if the $allow array is empty, any file with an extension listed in $deny array will be denied 
  $deny = array('php');                               


        if ($ok && ($source = @file_get_contents($location.$file))) {
          if ($lang == 'raw') {                        // ADD HERE
            $renderer->doc .= $source;                 // ADD HERE
          } else {                                     // ADD HERE
            if (!$lang) { $lang = isset($extensions[$ext]) ? $extensions[$ext] : $ext; }
            $title = ($title) ? "".$renderer->_xmlEntities($title).""
                               : "file: ".$renderer->_xmlEntities($file)."";
          
            $renderer->doc .= "
$title
";
            $renderer->code($source, $lang);
            $renderer->doc .= "
";
          }                                            // ADD HERE
          return true;
        } else {                                     

> How can I extend this to let the raw code be parsed as wiki markup? --//bastl, 04.06.08//

----

>[[http://wiki.erazor-zone.de/doku.php/doku.php?id=wiki:users:e-razor|E-Razor]]: Hi, nice plugin!
> Anyway, I've changed this allow-deny stuff a bit to use regular expressions.
> Those settings are also read from $conf (so you can add the plugin settings to your local.php).
> Another thing, i'm sure  would look much better too.
> That's the new [[plugin:source:mod_rules|source]]

----

> Hello all, here are some requests\\ 
>  1) make searchable, from within the wiki, included files\\ 
>  2) recursive and automatic adding of files in specified directory\\ 
> thats it... im sure it sounds easier to implement than it actually might be; but those features i believe would be **terrific**.\\ 
> atomi 2005/12/25

----

I added a bit debug for my file, maybe u like it...

// prevent filenames which attempt to move up directory tree by using ".."        
if ($ok && $location && preg_match('/(?:^|\/)\.\.(?:\/|$)/', $file)) {
    $ok = false;
    $reason = "you are not allowed to use ..";
  }

if ($ok && ($source = @file_get_contents($location.$file))) {

  if (!$lang) { $lang = isset($extensions[$ext]) ? $extensions[$ext] : $ext; }
  $title = ($title) ? "".$renderer->_xmlEntities($title)."" 
         : "file: ".$renderer->_xmlEntities($file)."";

  $renderer->doc .= "
$title
";
  $renderer->code($source, $lang);
  $renderer->doc .= "
";
  return true;
} else {
    if($reason) $renderer->doc .= "
Unable to display file "".$renderer->_xmlEntities($file)."": " . $reason . "
";
  else $renderer->doc .= "
Unable to display file "".$renderer->_xmlEntities($file)."": It may not exist, or permission may be denied.
 " . $location.$file . "
";


}

-----

I added start / end feature, now you can specify the start/endline source lang | tile | start | end > --- [[plugin:source:mod_linenumbers|code]]

hope it will be build in :)

greetings Michael Grosser

----

Great plugin! However, it doesn't seem to work well with the ODT exporter... Any idea to fix this?
   Sylvain Bonneau

----

Gorgeous! This can be used to do a **very very cool bridge to an arbitrary version control system**: Just source in the viewcvs-checkout uri (this is of course dependent of which system you use, but all major scm systems ( even darcs ) have some webinterface). Example


  


-- //bastl, 04.06.08//

----

Another approach to access a (locally accessible) svn is to access svn directly via exec("svn cat ...") as shown here. Just put it in the very beginning of the _getSource()-function:


      if (preg_match('/^file/',$this->location.$file)) {
        exec('svn cat '.escapeshellarg($this->location.$file), $source);

        //exec produces no newlines at the end of each line
        for($i=0; $ilocation.$file);
      }


No warranties for security-problems ...  -- //bastl, 06.06.08//


I put that code into the syntax.php, but I'm not understanding how to use the source function so that I can svn cat a file.  Can you add a line to show how I would use it?

//Jason    June 6, 2008//
----
\\ 
Could this easily be modified so that it only uses files that are stored via media manager? That would cancel out most security problems, wouldn't it?\\ --- //[[marg@rz.tu-clausthal.de|Christian Marg]] 2008/08/19 15:52//
----

==== Numbering Lines ====

This change in syntax.php will number the lines:\\ 
last four source lines


    function _getSource($file,$start,$end) {

      $source = @file($this->location.$file);
      if (empty($source)) return '';

      // $start is a 1 based index, need to correct to 0 based when slicing arrray
      if (!empty($start)) {
        $lines = count($source);
        if ($start > $lines) {
          $source = $this->getLang('error_start');
        } else if ($end < $start) {
          $source = $this->getLang('error_end');
        } else if ($end > $lines) {
          $source = join('',array_slice($source,$start-1));
        } else {
          $source = join('',array_slice($source,$start-1,$end-$start));
        }
      } else {
        $source = join('',$source);
      }

      $rows = explode("\n", $source);
      for($i = 0; $i Joe, 2013-01-10

===== Details =====

==== syntax.php  ====



 *   filename  (required) can be a local path/file name or a remote file uri
 *             to use remote file uri, allow_url_fopen=On must be set in the server's php.ini
 *             filenames with spaces must be surrounded by matched pairs of quotes (" or ')
 *   lang      (optional) programming language name, is passed to geshi for code highlighting
 *             if not provided, the plugin will attempt to derive a value from the file name
 *             (refer $extensions in render() method)
 *   title     (optional) all text after '|' will be rendered above the main code text with a
 *             different style. If no title is present, it will be set to "file: filename"
 *
 *  *** WARNING ***
 *
 *  Unless configured correctly this plugin can be a huge security risk.
 *  Please review/consider
 *    - users who have access to the wiki
 *    - php.ini setting, allow_url_fopen
 *    - php.ini setting, open_basedir
 *    - this plugin's location, allow & deny settings.
 *
 * @license    GPL 2 (http://www.gnu.org/licenses/gpl.html)
 * @author     Christopher Smith 
 */
if(!defined('DOKU_INC')) die();  // no Dokuwiki, no go

if(!defined('DOKU_PLUGIN')) define('DOKU_PLUGIN',DOKU_INC.'lib/plugins/');
require_once(DOKU_PLUGIN.'syntax.php');

/**
 * All DokuWiki plugins to extend the parser/rendering mechanism
 * need to inherit from this class
 */
class syntax_plugin_source extends DokuWiki_Syntax_Plugin {

  //------------------- [ Security settings ] ---------------------------------------------
  var $location = '';     // prepended to all file names, restricting the filespace exposed to the plugin
  var $allow = array();   // if not empty, ONLY files with the extensions listed will be allowed
  var $deny = array();    // if $allow array is empty, any file with an extension listed in $deny array will be denied
  var $rules = array();   // more complex allow/deny rules, refer documentation

  //------------------------[ Other settings ] ---------------------------------------------
  var $extensions = array(
      'htm' => 'html4strict',
      'html' => 'html4strict',
      'js' => 'javascript'
    );

    /**
     * return some info
     */
    function getInfo(){
      return array(
        'author' => 'Christopher Smith',
        'email'  => 'chris@jalakai.co.uk',
        'date'   => '2008-08-13',
        'name'   => 'Source Plugin',
        'desc'   => 'Include a remote source file
                     Syntax: ',
        'url'    => 'http://www.dokuwiki.org/plugin:source',
      );
    }

    function getType() { return 'substition'; }
    function getPType() { return 'block'; }
    function getSort() { return 330; }

    /**
     * Connect pattern to lexer
     */
    function connectTo($mode) {
      $this->Lexer->addSpecialPattern('',$mode,substr(get_class($this), 7));
    }

    /**
     * Handle the match
     */
    function handle($match, $state, $pos, Doku_Handler $handler){
      $match = trim(substr($match,7,-1));                    //strip  from end

      // ['|"]?[\1] [#-]  | 
      list($attr, $title) = preg_split('/\|/u', $match, 2);  //split out title

      $attr = trim($attr);
      $pattern = ($attr{0} == '"' || $attr{0} == "'") ? $attr{0} : '\s';
      list($file, $prop) = preg_split("/$pattern/u", $attr, 3, PREG_SPLIT_NO_EMPTY);

      if (isset($prop) && trim($prop)) {
        $matches = array();
        if (preg_match('/\s*(?:(?:#(\d+)-(\d+))\s*)?(\w+)?/',$prop,$matches)) {
          list(,$start,$end,$lang) = $matches;
          if (!isset($lang)) $lang = '';
        }
      } else {
        $start = $end = $lang = '';
      }

      return array(trim($file), $lang, (isset($title)?trim($title):''), $start, $end);
    }

    /**
     * Create output
     */
    function render($format, Doku_Renderer $renderer, $data) {

      $this->_loadSettings();

      list($file, $lang, $title, $start, $end) = $data;
      $ext = substr(strrchr($file, '.'),1);

      $ok = false;
      if (count($this->allow)) {
        if (in_array($ext, $this->allow)) $ok = true;
      } else {
        if (!in_array($ext, $this->deny)) $ok = true;
      }

      // prevent filenames which attempt to move up directory tree by using ".."
      if ($ok && $this->location && preg_match('/(?:^|\/)\.\.(?:\/|$)/', $file)) $ok = false;
      if ($ok && $this->rules) $ok = $this->_checkRules($file);

      if (!$lang) { $lang = isset($this->extensions[$ext]) ? $this->extensions[$ext] : $ext; }

      switch ($format) {
        case 'xhtml' :

          if ($ok && ($source = $this->_getSource($file,$start,$end))) {

            $title = ($title) ? "<span>".hsc($title)."</span>"
                               : $this->_makeTitle($file, $start, $end);

            $renderer->doc .= "<div class='source'><p>$title</p>";
            $renderer->code($source, $lang);
            $renderer->doc .= "</div>";
          } else {
            $error = sprintf($this->getLang('error_file'),hsc($file));
            $renderer->doc .= '<div class="source"><p><span>'.$error.'</span></p></div>';
          }
          break;

        case 'metadata' :
          if ($ok) {
            $renderer->meta['relation']['haspart'][$file] = array('owner'=>$this->getPluginName());
          }
          break;

        default :
          if ($ok) {
            $renderer->code($this->_getSource($file,$start,$end), $lang);
          }
      }

    }

    function _makeTitle($file,$start,$end) {
      $lines = $start ? sprintf($this->getLang('lines'),$start,$end) : '';
      $title = sprintf($this->getLang('title'),hsc($file),$lines);

      return $title;
    }

    function _getSource($file,$start,$end) {

      $source = @file($this->location.$file);
      if (empty($source)) return '';

      // $start is a 1 based index, need to correct to 0 based when slicing arrray
      if (!empty($start)) {
        $lines = count($source);
        if ($start > $lines) {
          $source = $this->getLang('error_start');
        } else if ($end < $start) {
          $source = $this->getLang('error_end');
        } else if ($end > $lines) {
          $source = join('',array_slice($source,$start-1));
        } else {
          $source = join('',array_slice($source,$start-1,$end-$start));
        }
      } else {
        $source = join('',$source);
      }

      return $source;
    }

    function _checkRules($file) {
      $permit = true;
      foreach ($this->rules as $rule) {
        list($allow_deny, $pattern) = $rule;
        if ($allow_deny == 'allow') {
          if (preg_match($pattern,$file)) $permit = true;
        } else {
          if (preg_match($pattern,$file)) $permit = false;
        }
      }

      return $permit;
    }

    function _loadSettings() {
      static $loaded = false;

      if ($loaded) return;

      $this->location = $this->getConf('location');

      $allow = $this->getConf('allow');
      $this->allow = !empty($allow) ? explode('|',$allow) : array();

      $deny = $this->getConf('deny');
      $this->deny = !empty($deny) ? explode('|',$deny) : array();

      $rules = $this->getConf('rules');
      if (!empty($rules)) $this->_parseRules($rules);

      $loaded = true;
    }

    function _parseRules($rules) {
      $rules = explode("\n",$rules);
      foreach ($rules as $rule) {
        $rule = trim($rule);
        if (!$rule || $rule{0} == ';') continue;

        $match = array();
        if (!preg_match('/^(allow|deny)\s+(.+)$/i',$rule,$match)) continue;

        $this->rules[] = array(strtolower($match[1]),$match[2]);
      }
    }
}

//Setup VIM: ex: et ts=4 enc=utf-8 :
</code>

 --- [[user>Juergen_aus_Zuendorf|Juergen_aus_Zuendorf]] //2018-02-06 17:27//\\
-> Changes for compatibility to PHP7:\\
"&$handler" and "&$renderer" to "Doku_Handler $handler" and "Doku_Renderer $renderer"
==== action.php ====

<code php>
<?php
/**
 * Source Plugin: includes a source file using the geshi highlighter
 *
 * Action plugin component, for cache validity determination
 *
 * @license    GPL 2 (http://www.gnu.org/licenses/gpl.html)
 * @author     Christopher Smith <chris@jalakai.co.uk>
 */
if(!defined('DOKU_INC')) die();  // no DokuWiki, no go

if(!defined('DOKU_PLUGIN')) define('DOKU_PLUGIN',DOKU_INC.'lib/plugins/');
require_once(DOKU_PLUGIN.'action.php');

/**
 * All DokuWiki plugins to extend the parser/rendering mechanism
 * need to inherit from this class
 */
class action_plugin_source extends DokuWiki_Action_Plugin {

    /**
     * return some info
     */
    function getInfo(){
      return array(
        'author' => 'Christopher Smith',
        'email'  => 'chris@jalakai.co.uk',
        'date'   => '2008-08-13',
        'name'   => 'Source Plugin',
        'desc'   => 'Include a remote source file',
        'url'    => 'http://www.dokuwiki.org/plugin:source',
      );
    }

    /**
     * plugin should use this method to register its handlers with the DokuWiki's event controller
     */
    function register(Doku_Event_Handler $controller) {
      $controller->register_hook('PARSER_CACHE_USE','BEFORE', $this, '_cache_prepare');
    }

    /**
     * prepare the cache object for default _useCache action
     */
    function _cache_prepare(&$event, $param) {
      $cache =& $event->data;

      // we're only interested in wiki pages and supported render modes
      if (!isset($cache->page)) return;
      if (!isset($cache->mode) || in_array($cache->mode,array('i','metadata'))) return;

      $max_age = $this->_cache_maxage($cache->page);
      if (is_null($max_age)) return;

      if ($max_age <= 0) {
        // expire the cache
        $event->preventDefault();
        $event->stopPropagation();
        $event->result = false;
        return;
      }

      $cache->depends['age'] = !empty($cache->depends['age']) ?  min($cache->depends['age'],$max_age): $max_age;
    }

    /**
     * determine the max allowable age of the cache
     *
     * @param   string    $id         wiki page name
     *
     * @return  int                   max allowable age of the cache
     *                                null means not applicable
     */
    function _cache_maxage($id) {
      $hasPart = p_get_metadata($id, 'relation haspart');
      if (empty($hasPart) || !is_array($hasPart)) return null;

      $location = $this->getConf('location');

      $age = 0;
      foreach ($hasPart as $file => $data) {
        // ensure the metadata entry was created by or for this plugin
        if (empty($data['owner']) || $data['owner'] != $this->getPluginName()) continue;

        $file = $this->getConf['location'].$file;

        // determine max allowable age for the cache
        // if filemtime can't be determined, either for a non-existent $file or for a $file using
        // an unsupported scheme, expire the cache immediately/always
        $mtime = @filemtime($file);
        if (!$mtime) { return 0; }

        $age = max($age,$mtime);
      }

      return $age ? time()-$age : null;
    }

}

//Setup VIM: ex: et ts=4 enc=utf-8 :
</code>

===== CSS =====

The plugin comes with some style additions in the ''style.css'' file. 
  * The style colouring, green, has been chosen to complement but differentiate ''%%<source>%%'' blocks from the ''%%<code>%%'' (blue) and ''%%<file>%%'' (red) blocks rendered by the [[plugin:code|code]] plugin.  
  * The styling itself is similar to and inspired by that used by [[http://gentoo-wiki.com|gentoo-wiki]], a media wiki installation.

These can of course can be modified to suit your own requirements.  More details on modifying the styles are given after the file contents listing.

**style.css**

<code css>
/*
 * source plugin extension - style additions
 *
 * @author  Christopher Smith  chris@jalakai.co.uk
 * @link    http://wiki.jalakai.co.uk/dokuwiki/doku.php/tutorials/codeplugin
 */
/* source plugin extensions */

/* layout */
div.source {
  width: 92%;
  margin: 1em auto;
  border: 1px solid;
  padding: 4px;
}

div.source p {
  font-size: 90%;
  margin: 0;
  padding: 2px;
}

div.source p span {
  font-weight: normal;
}

div.source pre.code {
  margin: 4px 0 0 0;
}

/* colours */
div.source {
  border-color:  #bdb;
  background: #e4f8f2;
}

div.source p {
  background: #c4e4d4;
}

div.source pre.code {
  border: 1px dashed #9c9;
  background: #ecfaf6;
}
</code>

The source file contents are wrapped within a ''<div>'' tag of class ''source'' and the file name is given above the contents.  The html fragment will look like

<code html>
<!-- with title -->
<div class='source'>
  <p><span>{title}</span></p>
  <pre class='code {language}'>{file contents}</pre>
</div>

<!-- with no title -->
<div class='source'>
  <p>file: <span>{filename}</span></p>
  <pre class='code {language}'>{file contents}</pre>
</div>
</code>
 \\
Therefore, standard DokuWiki styling can be overridden with the following style selectors:
<code css>
/* style title and "file: {filename} */
/* set to display:none to prevent it being displayed at all */
div.source p

/* style title and {filename} only (not "file:") */
div.source p span

/* style the file contents, e.g. font, border & background */
  div.source .code {} 

/* override the geshi hilight styles in /lib/styles/style.css */
  div.source .code .<geshi-hilite-code> {} 
</code>

\\
==== Tab size ====
You can set the tab-size to 4 as the default, for Chrome, Firefox and Opera, using:
<code css>
div.source pre.code {
  tab-size: 4;
  -moz-tab-size: 4;
  -o-tab-size: 4;
}
</code>
===== Bugs =====