Making files

Make files, phew!? Admitted I previously used the shell approach by adding mostly bash commands in the files I made. However, I somehow felt I had to change that, since making make files generic and not utilising the full potential of make if both hard and rather stupid.

I have a repository setup for Debian packages, these packages are of two types, some are simple configuration packages and others are Debian packages made from installation ready 3rd party tarballs. I aim at wrapping these 3rd part packages into Debian packages for easy usage for all my repository users.

The setup for creating the packages are a top level directory called packages, which contain the source for each of the individual packages. These could be Altera, a package that contain a setup for the Altera suite or Slickedit, a package containing the setup for Slickedit used in our users development environment.

The common rule set for all the individual package directories are:
1. Each directory must be named after the package i.e Slickedit package is in a directory called Slickedit
2. Each directory must contain a Makefile
3. Each directory must contain a readme.asciidoc file (documentation x3)
5. Each directory must contain a tar.gz file with the source package
4. Each directory must contain at least one source directory appended with -package

The above rules gives the following setup for the Slickedit package:

#tree -L 1 slickedit/
slickedit/
├── Makefile
├── readme.asciidoc
├── slickedit-package
└── slickedit.tar.gz

1 directory, 3 files

The package make file targets from the system that utilizes the package build routine are all, clean, distclean. all extract and builds the Debian package, clean removes the Debian package and distclean removes the extracted contents of the -package directory.

Furthermore, the make file contain three targets named Package, pack and unpack where package builds the debian package from the -package directory, pack creates a tarball for the -package directory in case there are changes to the package source, unpack extract the tarball into the -package directory.

Make file for the packages:
DEBCNTL = DEBIAN
TARFLAGS = -zvf
EXCLUDES = --exclude $(DEBCNTL)

# These are the variales used to setup the various targets
DIRS = $(subst /,,$(shell ls -d */))
PKGS = $(shell ls *.tar.gz)

CLEAN_DIRS:=$(subst package,package/*/,$(DIRS))
DEBS:= $(DIRS:%-package=%.deb)
TGZS:= $(DIRS:%-package=%.tar.gz)

# These are the targets provided by the build system
.PHONY: $(DIRS)

all: unpack package

package: $(DEBS)

pack: $(TGZS)

unpack: 
find . -name '*.tar.gz' -type f -exec tar -x $(TARFLAGS) "{}" \;

clean:
rm -vf $(DEBS)

distclean: clean
ls -d $(CLEAN_DIRS) | grep -v $(DEBCNTL) | xargs rm -fvR 

# These are the stem rules that set the interdependencies
# between the various output types
$(DEBS): %.deb: %-package

$(TGZS): %.tar.gz: %-package

# Stem targets for generating the various outputs
# These are the commands that generate the output files
# .deb, .tar.gz and -package/ directories
%.deb: %-package
fakeroot dpkg-deb --build $< $@

%.tar.gz: %-package
tar -c $(TARFLAGS) $@ $< $(EXCLUDES)

The major pit fall I had when creating the make file was figuring out the rules for the .tar.gz .deb and
-package rules. The first two are straight forward to create using the % modifier, but when creating the
-package target I ran in to a circular dependency. Because the pack and unpack rules have targets that when defined using the static pattern rule are contradictory.

%.tar.gz: %-package 

and the opposite

%-package: %.tar.gz 

Caused the unpack target to execute both the extract and create tarball targets, leading to the error of a Debian file containing only the Debian directory, not much package there. Since the aim was a generic make file, one to be used for all packages, I ended up using the find command to find and extract all tarballs. I figures this was the easiest approach since using the static pattern rules didn't work as intended.

Custom spotify playlists

The problem with spotify and iTunes etc is that there's really no good support for playlists. With playlists I mean importing say csv lists. Why would i need a list like this? Because, every now and then I'd like to just create a simple playlist containing the Artis Name and track number as ARTIST,TRACK repeat.

That's where ivyishere comes in, cool online tool that can export your csv, amongst other, playlists to spotify. Thanks.

How do I get a csv playlist, in this case I found the work of another House MD fan who did spend the time to organize all tracks from all shows on a website. I'd like that playlist without hving to add every single song by hand.

This i what I did.

1. copy the neatly ordered html table using you mouse, from top till bottom, selecting all the songs you like in your playlist.

2. open libreoffice calc and paste the songs there, select the html formatted table and wait.

3. Delete the rows you dont need, keeping only artist and track.

4. Copy the artist and track coloumns paste in a new document, then save this document as csv.

5. Fire up your shell and perl the shit outta a the csv.

cat housemd.playlist.csv  # to see what you have to deal with

Figure out the reguar expressions you'll need, yes you can most likely find a more combined expression than I did, nevertheless my way worked ;)


perl -p -e "s/^,$//g" # Remove the lines containing just ,
perl -p -e "s/^\s$//g" # remove lines containing just whitespaces
perl -p -e "s/^'No.*,$//g" # remove th elines containing the No Commer ... text
perl -p -e "s/\"|\'//g" # remove the ' and " from all artists and songs

The final experssion for me was:

perl -p -e "s/^,$//g" housemd.playlist.csv | perl -p -e "s/^'No.*,$//g" | perl -p -e "s/^\s$//g" | perl -p -e "s/\"|\'//g" > housemd.playlist.2.csv

6. Cat your file to see if it is what you'd expect

7. Upload you file to ivyishere and wait.

Thanks perl and ivyishere.

Asciidoc filters

Using asciidoc? No?! How come? Because if you're writing technical documentation, blogging or books, this is one of the best programs available.

Asciidoc is installed using you distribution or systems package manager i.e for MacPorts on OSX:

$ sudo port install asciidoc

Substitute port with apt-get, aptitude, yum or whatever the package manager on your system is. Now that asciidoc is installed, simply start writing your document, blog or book in a text file in your favorite editor. Open it and paste the following text.

= My first Asciidoc

:author: <your name>

:toc:

== Introduction

When you're ready to release or upload. Generate an neat looking document using the command:

.Generating an asciidoc

[source,sh]

asciidoc my.first.asciidoc;

To see the document that you have just created, simply follow the document's instructions and run the command, with your document name obviously.

$ asciidoc my.first.asciidoc

The following renders like this using the standart asciidoc theme:

To get started quickly with asciidoc Powerman has created cheat sheet, use this to see some of the things you can do. one of the things not included in the cheat sheet is the fact that asciidoc allows you to execute scripts on the command line.

This is extremely useful when writing technical documentation where you'll need to include information like realtime directory contents information or reporting logged in users directly in your document. 

Adding a shell command ensures that the command is run when the document is being generated, every time the document is generated. Add the following to your document.

.Listing asciidoc's present in the current directory

[source,sh]

----

sys::[ls *.asciidoc]

----

Now, instead of rendering the actual command, asciidoc executes the command and renders the result. In this case the normal output of the /ls *.asciidoc/ command. 

As you can see I have 2 asciidoc files in my current directory. If I wanted to I could include the filter.asciidoc file in the one I'm currently writing. Allowing the file filters.asciidoc to be included. Add the following text to your asciidoc file:

Adding the next document after this line

:leveloffset: 1

include::filters.asciidoc[]

:leveloffset: 0

And were back in the original document.

The include::<filename>[] statement is where the magic happens. This is the line that includes the filter.asciidoc file. The included file doesn't have to be an asciidoc document, any file can be used.

The :leveloffset:1 is needed for asciidoc to treat the included document as headers in its header level. After the include statement we simply pop the header level bask with :leveloffset:0 And were back in the original document.

Notice how the included document is present with in the Adding ... and were back in text. A very useful feature when you have more than one person working on documentation. As this avoids numbers of document merges when you're working on separate files in a team.

Did you notice the cool graphics that was included in the document? This graphic is rendered by an asciidoc plugin called Ditaa.  There are several plugins available for asciidoc, some these can be found on the asciidoc plugin page.

Each of the plugins have installation instructions and usage information included. Here's an example of the ditaa plugin. First download the ditaa.zip file. Then install it to your local users ~/.asciidoc directory.

$ mv Downloads/asciidoc-ditaa-filter-master.zip Downloads/ditaa-filter-master.zip

$ asciidoc --filter install Downloads/ditaa-filter-master.zip

The ditaa plugin is the one rendering the image displayed in the my.first.asciidoc. Here's how it's done. Create a new file called filter.asciidoc and fill it with these contents.

= Asciidoc filters exmple

:author: John Dideriksen

:toc:

== testing various filters on the MacPort edition

The following document is used to test some of the asciidoc plugins for drawing, all examples have been taken from the authors plugin documentation page.

=== Ditaa

["ditaa"]

---------------------------------------------------------------------

    +--------+   +-------+    +-------+

    |        | --+ ditaa +--> |       |

    |  Text  |   +-------+    |diagram|

    |Document|   |!magic!|    |       |

    |     {d}|   |       |    |       |

    +---+----+   +-------+    +-------+

        :                         ^

        |       Lots of work      |

        +-------------------------+

---------------------------------------------------------------------

stops here

As you can see ditaa renders the graphic from an ascii image. This is really useful since you do not have to worry about opening a new program and maintain a separate drawing.

If your documentation is under source control you can easily track the changes in the diagrams for the documentation, just like any other changes. Ditaa is just one of many filters you can install in asciidoc, aafigure is another example of a filter.

You can list your installed asciidoc filters using the command:

$ asciidoc --filter list

/opt/local/etc/asciidoc/filters/code

/opt/local/etc/asciidoc/filters/graphviz

/opt/local/etc/asciidoc/filters/latex

/opt/local/etc/asciidoc/filters/music

/opt/local/etc/asciidoc/filters/source

/Users/cannabissen/.asciidoc/filters/ditaa

Notice, that asciidoc comes with a set of preinstalled plugins that you can use at your will. You remove an installed filter with the command:

$asciidoc --filter remove ditaa

The asciidoc files for this short example can be found downloaded from here: my.first.asciidoc & filters.asciidoc.