Any science student has, at some point, sat through a lecture on research methods which covers reliability and validity as factors which must be taken into account. They can be handily summarised as: ‘If I, or someone else, repeat this experiment, will it give the same sort of result?’, and ‘Is my experiment actually measuring what I think it’s measuring?’. Obviously a well-designed experiment will fulfill both these conditions. Usually a series of funny little examples are presented, in the hope they’ll stick in your memory and help you distinguish between the two. Despite this, I’d mostly forgotten about such things, until I saw the cartoon below – a particularly good example of an experiment high in reliability that isn’t measuring what it’s meant to be measuring – and the sort of result that neuroscientists really don’t want to see!
(source:) http://xkcd.com/1453/ Used under Creative Commons Attribution-NonCommercial 2.5 License.
If you’d like to find out more information about fMRI, psychcentral.com has a very good summary here.