Processes, procedures, and reference
Loading...
Searching...
No Matches
The Pixo Documentation System

For activated github repositories, developers can push/merge to the main branch of a repository to trigger a documentation build of a repo.

This is useful because documentation is updated frequently and matches the existing state of the main branch. It also allows developers to add their own example pages and explanations at will.

The Flow

To create this linkage:

  1. A documentation method is added to a repository. This will typically be docs-doxygen, which is based on Doxygen. It supports C, C++, C#, Python, Java, XML, and many other languages. Support for Go and Javascript are planned using other methods.
  2. The env.sh file is updated to name the documentation, add an icon, point to source code, and provide any other needed flags for publish.
  3. A request is made to our devops team to add the repository to a known url and to add the CI hooks needed to trigger a publish.
  4. The repository then needs to be added to docs-main so it can appear as an item in the main documentation menu. This will need to be run locally and then committed and pushed, probably by an administrator. You will want to softlink it into a folder like Unity/ or Unreal, but it can be any folder. The name of the folder may appear in the main menu as the design changes.
  5. In the simple case of parsing source code, this is all that is required. For the less simple case of wanting to publish blueprints, materials, or whole projects from Unreal, see https://github.com/PixoVR/docs-doxygen/blob/main/README.md
  6. From here, any push to main on the repo will trigger an update to its documentation.

Example:

git clone https://github.com/PixoVR/somerepo.git
cd somerepo
mkdir documentation
cd documentation
git submodule add https://github.com/PixoVR/docs-doxygen.git
./docs-doxygen/setup_submodule.sh # initialize local copies of the template into your repo
# ask your administrator to activate the repository on main,
# and provide the folder you want it published to.
cp /some/repo/icon/from/somewhere/transparentIcon_128x128.png ./
vim env.sh # change the title, source directories, icon, description, subfolder from administrator, etc.
vim pages/mainpage.md # write some documentation here, and add pages as needed.
vim style.css # modify style as needed
# if you are trying to publish blueprints and materials, copy the right Dockerfile to this repo
#cp docs-doxygen/setup_template/Dockerfile-unreal4-plugin ./Dockerfile
#vim env.sh # update the UNREAL_UFS_PATHS and UNREAL_PROJECT
git add .
git commit -a -m "documentation added"
git push origin main # only the main branch should be documentation active
# documentation should build and appear shortly at the url.
# modify docs-root to include your menu item, build.sh, git add, and the push docs-root to main

Components and Repositories

docs-doxygen

https://github.com/PixoVR/docs-doxygen

This is the Doxygen publisher, to be included in documented repositories. This has special parsing modes for Unreal and Unity, which adds some nice information about metadata, blueprints, serializability, etc. The parsers for this are found in the scripts folder, and are linked through the Doxyfile.

This uses Doxygen Awesome for layout and styling.

Generally it is dangerous to modify the global styling because a small change can affect a lot of different elements. It is preferrable to not change the master global repository docs-doxygen.

The Pixo styling is comprised of these parts:

  • doxygen-custom/doxygen-pixo.css is the main style, which overrides doxygen-awesome.css and a few other css files. Don't modify this unless you know what you're doing.
  • doxygen-custom/header.html and doxygen-custom/footer.html wrap every page and initialize some javascript. Note that even though these files have an .html extension, there are tokens in them that are parsed by doxygen.
  • doxygen-custom/headermenu.js - The upper left menu widget is created here using select2.min.js. The menu.json file that it looks for is created by docs-root.
  • style.css is found in your repository after running setup_submodule.sh. For style changes to a particular repository, do not modify doxygen-pixo.css. Instead use this style.css file.
  • Additionally, css file(s) can be included in env.sh using the DOXYGEN_CSS variable.
  • Doxyfile controls the Doxygen parser. It can be useful to update the scripts/, or to change call/callee/uml settings, as well as add custom file formats/parsers. It is not preferrable to modify this file for a single repository. It's better to copy it into your documentation folder and then in build.sh reference it with doxygen ../Doxyfile. build.sh will cd into docs-doxygen, and therefore paths are relative to it.
  • nginx/ controls the nginx server created for this repo.
  • nginx/errors/error.html is the general purpose error page, which is necessarily composed of a single self-contained file that does not reference anything else. This is where the 404 graphic comes from for bad url requests.
  • dot_wrapper.sh is a wrapper around the dot command, because we have to fix the bounding area of dot files generated from blueprints.
  • setup_*.sh are called by Dockerfile or during initial repo setup.
  • setup_template/ contains the skeleton versions of files to be copied into your repository. These don't affect your project, but instead are copied into your repository so they can be changed. The intention is that the documentation settings are unique to a repo, and are a derivative of this template.

Here's an example of the preferred method for adding a special section or treatment to a repo's documentation:

# wrap sections of your code in a div with a class or id
<div class='myuniqueclass'>
Here is some text that I want to look different for some reason.
+---+---+---+
| A | B | C |
+---+---+---+
</div>
/* add styles under that class or id */
.myuniqueclass {
text-align: center;
}
.myuniqueclass table {
display: inline-block;
background-color: purple;
font-size: 4em;
}

Please Note: Because this is a central library to the whole documentation system, it is made public on GitHub. A variety of things clone it, all at various stages of authentication or automation, and the expectation is that a change to this repo can be propapgated simply by executing a rebuild, and other repos that use it will do a git pull main to stay up to date, usually in a Dockerfile.

docs-root

https://github.com/PixoVR/docs-root

This generates the root menu structure. Basically I take this repo, softlink any repos that want to be in the main menu into their respective subfolders at the root, and then run ./build.sh from the documentation folder to generate the main menu. The content is then committed to the repo and merged to main.

There's not much to this, but here are the files:

  • build.sh creates the menu structure on your local system by searching for env.sh files in subfolders of this repository and following softlinks. If you want it to work on a local system, you can use DEV=true ./build.sh, which basically just uses the DEV_ version of variables. This is rarely needed, and is a little annoying because you may have to update the DEV_ variables for every repo to be useful.
  • menu/index.php is the menu generator script. It does the searching and parsing. This also produces the menu.json file which is needed for the upper left menu found in all sub pages. This short script is a little tricky in how it reads variables from env files.
  • menu/style.css is the css file for the main menu, which is separate from style.css which is for the documentation pages (like this one) that come with the main menu.
  • pages/ contains this page. If you're reading this page in a browser, is generated from the pages/pixodocs.md file. Other files can be added to this folder and menu.md.

This whole system relies upon urls to link it all together. Each repo is a separate nginx server, and the URL convention links it all together. docs-root is the menu linkage between the urls. This allows each one to publish separately without depending on each other, and allows each repo to have its own settings, language, access, or whatever we want.

Building on a Local System for Preview

Once you have added docs-doxygen to a repo and you have installed doxygen and graphviz, you can view documentation on your local system before publishing it.

sudo apt install doxygen graphviz
cd documentation
./build.sh

Now you can view the files created in documentation/html either as flat files from the filesystem, or if you have a local webserver running, you can point it to that folder.

If you are on a Windows system, you can run the above commands in WSL bash as well.

docs-docker-base

https://github.com/PixoVR/docs-docker-base

This repo creates the base docker image that all documentation pulls from when building on GCloud.

Please Note: This repository creates an authenticated docker image on GCloud, and because it is used as a base image, other things on top of it can use this authentication for git operations on GitHub. At its core, a Personal Access Token is generated on GitHub, and that token is applied in the secretManager env GIT_TOKEN. It is currently set to never expire, but that is easy enough to change by generating a new key, or revoking the current one.

  • Dockerfile is used for the build, and does an apt install of lots of software needed to publish with Doxygen. It clones docs-doxygen into the environment, then does a source code build of graphviz and doxygen. I suppose those build scripts could be moved here instead of docs-doxygen, but with it the way it is, docs-doxygen is all that is needed to build a documentation artifact on a local system.
  • cloudbuild.yaml is the CI/CD cloudbuild file for GCloud. Building from source for Doxygen and Graphviz takes a long time, and benefits from a larger machine like E2_HIGHCPU_32. This is also where the secret GIT_TOKEN is introduced to the build process.
  • Makefile can be used on a local system to test the base image. It is not used anywhere else in the documentation process.

docs-docker-ue4-base

https://github.com/PixoVR/docs-docker-ue4-base

This repo uses the Personal Access Token mentioned above to pull the Unreal 4.27 image from Epic Games, and in turn build an image on top of it that contains an Unreal 4.27 project skeleton used for documenting blueprints and materials.

Please Note: Whatever user generates the Personal Access Token must also have access to the Epic Games repo https://github.com/epicgames for this to work the first time. After that, we maintain/cache an Unreal image on our cloud. If the data center is moved or changed, this will have to be re-cached. After the cache is complete, for instance, a different Personal Access token can be used that does not have Epic Games access.

  • cloudbuild.yaml calls the Dockerfile to package the skeleton project and create a base image for Unreal documentation builds. If this is being run for a first time in an environment, it will cache the Unreal image to GCloud for future use. This step will be very slow, but is much faster on later runs.
  • Dockerfile builds and freshens the base image docs-docker-ue4-base. This file receives the GIT_TOKEN value needed for future authentication in the image.

pixo-unreal-documentation

https://github.com/PixoVR/pixo-unreal-documentation

This is an Unreal plugin that parses .uasset files and produces fake C++ code for Doxygen. For blueprints and materials to appear in a Doxygen publish, this must be run manually or via commandlet, which the Dockerfile-unreal4-plugin script does.

This is raw source code, and is compiled during the cloud build in a Linux environment. In some future release it will also parse Behaviour Trees, Animations, and all the other graph-able things in Unreal.

docs-pkgsite

https://github.com/PixoVR/docs-pkgsite

Currently unused, but may be used in the future to document projects written in Go. The installation method is similar to docs-doxygen.