Log in

Q3A with open source generated shaders! - LIBV Intentionally Breaks Videodrivers [entries|archive|friends|userinfo]
Luc Verhaegen

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Q3A with open source generated shaders! [Mar. 18th, 2013|08:28 pm]
Luc Verhaegen
[Tags|, , , , , , , , , ]
[Current Location |Rumpelkammer]
[mood |Happy]
[music |Cornershop - Brimful of asha (norman cook remix)]

The combination of limare and open-gpu-tools can now run Quake 3 Arena timedemo without depending on the binary driver for the shader compiler!

Connor Abbott has been being his amazing (16y old!) self again in the weeks after his talk at FOSDEM, and he pushed his compiler work in his open-gpu-tools tree to be able to handle basic vertex shaders. Remember that our vertex shader is a rather insane one, where the compiler has to work real hard on getting scheduling absolutely right. This is why an assembler for our vertex shader was not too useful and the most part of a compiler had to be written for it to generate useful results. A mammoth task, and Connor his vertex shader code is now larger than the code I have in my limare library.

So it was high time that we brought limare and OGT together to see what they were capable of with some basic shaders. Luckily, the Q3A GLES1 emulation has basic shaders, what a nice coincidence :)

So Connor turned my simple vertex shader essl into the high level language used by the OGT vertex shader compiler, and through steps described at this wiki page, turned them into MBS files (Mali Binary Shader - the file type output by the standalone compiler, and also by newer binary driver integrated compilers). Limare can then load and parse those MBS files, and run the shaders. No need to involve the ARM binary anymore when we have OGT generated MBS files :)

The result was quite impressive. We had a few issues where the limare driver (which has mostly taken its cues from the output of the binary driver) and OGT disagreed over symbol layout, but apart from that, bringing up the shaders connor produced was pretty painless. Amazingly effortless, for such a big step.

Connor then spent another day playing with the fragment shader assembler, fixed some bugs, and produced 3 fragment shaders for us. One for the clear shader used by limare directly, and 2 for Q3A. After some more symbol layout issues, these also just worked! We even seem to be error-margin faster with the MBS files (due to texture coordinate varyings being laid out differently).

So this is a really big milestone for the lima driver project. Even with our insane pre-optimized architecture, we now are able to run Quake 3 Arena without any external dependencies, and we are beating the ARM binary while doing so.

For generating your own shader MBS files, check out Connors OGT, and then you can head straight to Connors wiki page. My Q3A tree now has the MBS code included directly. And i pushed a dirty version of my FOSDEM limare code.

As for this new limare code, this fosdem_2013_pile branch will vanish soon, as i need to properly pry things apart still. This is run-for-the-price code, and often includes many unrelated fixes in the same commit. It's better to do archeology on it now, than 3y from now, so this needs to be split. But in the meantime, you all can go and give Q3A on a fully free driver stack on Mali hw a go :)

I will not post a video, as there really is nothing new to see. It is the exact same timedemo, running some promille faster. Build things, and then run it yourself on your sunxi hardware (i am still working on porting it to the new kernel of a more powerful platform). That's the best proof there is!

For building limare, check out the fosdem2013_pile branch and then just run make/make install.

For building Q3A all you need to do is run:
make ARCH=arm USE_LIMARE=1
And, when you have the full quake installed in ~ioquake3/baseq3, you can create a file called ~ioquake3/baseq3/demofour.cfg with the following content:
cg_drawfps 1
timedemo 1
set demodone  "quit"
set demoloop1 "demo four; set nextdemo vstr demodone"
vstr demoloop1
You can then run the ioquake3 binary with "+exec demofour.cfg" added to the command line, and you will have the demo running on top of fully free software!

Now we really have covered all the basics, time to find out how Mesa will play with our plans :)

From: (Anonymous)
2013-03-19 03:44 pm (UTC)
..... nice but we still cant use/test anything, shaders or not..... is a modification to make these drivers a decent substitute of mali-drivers planned? For when?
(Reply) (Thread)
[User Picture]From: libv
2013-03-19 03:47 pm (UTC)
Did you read the full blog entry?

This is a rather hardcore reverse engineering project, done in our spare time, things do not happen overnight, and you should not expect this to be otherwise.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-03-22 01:45 pm (UTC)
Hehe, this happens everytime.... but when will it be ready. Luc the only thing I do hope is that this will one day be ready. It's a pity to see so many open source projects start and then end without producing any visible end-user results.
(Reply) (Parent) (Thread)
[User Picture]From: libv
2013-03-22 02:03 pm (UTC)


Thanks for your contribution.
(Reply) (Parent) (Thread)
[User Picture]From: Marius Cirsta
2013-03-24 07:54 pm (UTC)
You're wellcome :).
I do realize you and the other devs are committed to your work so don't take it personally.
And I do contribute to open source projects (I'm a developer for a Linux distro) so I know how it is when end users expect this and that.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-05-01 07:33 pm (UTC)
Links to (substantial) contributions, or I call bullsh1t. No one who has said experience would utter such clueless, unappreciative words.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-09-15 01:13 pm (UTC)
is the binary mali driver that bad?
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-04-11 09:39 am (UTC)
Stop for a moment and consider that he's doing this work pro bono. Developer time is expensive and industry expert time even more so.

Think yourself lucky that you live in a world where such people care about giving their time to working on such causes. He probably has to listen to pundits like you constantly -- and he *still* keeps on working on the project anyway.

P.S. instead of hoping try *helping*.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-04-15 12:56 pm (UTC)
Uhm... yes, you can test something. He provides all the build steps to be able to run quake3 without proprietary drivers on a Mali GPU. That's it. If people would help testing and developing instead of bitching and waiting, open source GPU drivers would be running circles around the closed drivers by now. There is no magic involved.
(Reply) (Parent) (Thread)