Since some time there has been discussions about having a place for community reviews; see for example
The idea is to have a transparent peer-review process. Everything should work as if it were a standard journal, except for the fact that submissions are linked to papers in arXiv and reviews are public (but reviewers may be anonymous). The main task is to gather a community big enough for the project to work, but specialized enough so that submitted papers have a properly sized audience.
Any comments are welcome!