You might look into digital signing. Digital signing works like this:
you receive a file F
F contains a note saying that this file comes from person Y. (This note is called a signature.)
Person Y's note contains a second note from person X saying that person Y is telling the truth about who they say they are. (This note is called a CA signature.)
We can use cryptography to verify that, if you already have some information person X wrote and sent to you, then you can prove person X's note wasn't forged. (Specifically: you can prove that there's a secret number person X knows, that it matches a number X gave you a long time ago, the person who wrote the note knew X's number, and that you can't figure out what X's number is just by looking at the note.) Then you know that, to the best of person X's knowledge, person Y is telling the truth too. You can use person Y's information to prove that the file was not changed since it was sent, so if you trust person Y, you can open the file.
The historical problem with these systems is that person X is often fooled and person Y often mistakenly gives away secret information that lets other people forge his signature.
I don't know what "encrypt the code in a fractal manner" means but there are a lot of programs that look at their own code while running to make sure it hasn't been changed. There aren't as many programs that rely on cryptography to do this, because a person who modifies the program itself can also modify the cryptography information that the program uses to check if it has been modified. Digital signing is a pretty strong tool against people modifying the file on the hard drive, but it isn't a good defense against code that is already running, and it relies on assuming that something will actually check that the digital signature is correct. A hacker could make it so the program checks if the hacker signed it, rather than checking if the developer signed it. Some processors now have the ability to run encrypted code using a hidden region of memory not visible to the operating system or debuggers, and this might make it easier to prevent people from modifying your program. I don't know very much about this feature though.
I think your system sounds more complicated because it relies on making changes to the structure of file F. Also, knowing that a program came from a specific compiler probably isn't enough to determine that the program is safe to run. A compiler doesn't know the difference between malware and non-malware, and it's very hard to keep a software tool from getting into the hands of people who will use it for malicious things, because software is easy to copy. If your scheme was made insecure (for instance, if someone leaked the compiler) it might be hard to make people's computers stop treating "it came from moyo's compiler" as proof that the program is secure.
You might still try to throw together an implementation of your system to see if you get any interesting results along the way, though.
