Thanks for the advice. Thankfully I discovered how to printk early in. I'm running Linux 3.2.2 i386 on a VM and then I'm ssh'ing in. Everything is set up so that I code on my sys_xcrypt.c and xcipher.c file, run my makefile, sh install_module.sh to unload/load the module, and then ./xcipher blahblah to test it.
I've copied all the data I need from user land into kernel land, and my file IO works correctly. Pretty much all that's left is trying to figure out how to encrypt/decrypt with aes.
Oh and figuring out how to remove partial output files. (I was thinking vfs_unlink, but if I filp_close before or after then I oops.)
And figuring out why "filpout = filp_open(kargz->outfilezname, O_WRONLY | O_CREAT, 00666);" creates a file with user wr, group r, other r permissions. I was copy pasting the infile permissions from its inode struct instead of just hardcoding numbers, but I noticed the permissions weren't actually copying over. edit: Interesting. I can change user r/w/x, group r, others r/x, but it won't let me change group w/x or others w. From the system call that is, if I just chmod everything changes fine.
And then I need to add some extra error checking.. but I think I know what I'm doing there.
Edited, Dec 9th 2012 2:29pm by Deadgye