The consensus on diet and cancer is that obesity increases the risk of developing cancer. Particular dietary practices often explain differences in cancer incidence in different countries (e.g. gastric cancer is more common in Japan, while colon cancer is more common in the United States). Studies have shown that immigrants develop the risk of their new country, suggesting a link between diet and cancer rather than a genetic basis.
Despite frequent reports of particular substances (including foods) having a beneficial or detrimental effect on cancer risk, few of these have an established link to cancer. These reports are often based on studies in cultured cell media or animals. Public health recommendations cannot be made on the basis of these studies until they have been validated in an observational (or occasionally a prospective interventional) trial in humans.
The case of beta-carotene provides an example of the necessity of randomized clinical trials. Epidemiologists studying both diet and serum levels observed that high levels of beta-carotene, a precursor to vitamin A, were associated with a protective effect, reducing the risk of cancer. This effect was particularly strong in lung cancer. This hypothesis led to a series of large randomized trials conducted in both Finland and the United States (CARET study) during the 1980s and 1990s. This study provided about 80,000 smokers or former smokers with daily supplements of beta-carotene or placebos. Contrary to expectation, these tests found no benefit of beta-carotene supplementation in reducing lung cancer incidence and mortality. In fact, the risk of lung cancer was slightly, but not significantly, increased by beta-carotene, leading to an early termination of the study. 
However, Randomized Clinical Trials (RCTs) also have drawbacks in cancer prevention, particularly in micronutrient deficiencies, which are thought by some to be a major contributor to cancer. RCTs involve huge numbers of people, take many years to complete, and are therefore extremely expensive and complicated, and therefore few are done. In addition, these randomized clinical trials usually test only a single dose. An alternative, which is likely to be more useful, is to do shorter intervention trials focusing on other endpoints related to cancer, such as DNA damage. These trials can test a variety of doses on fewer people to determine what level of micronutrient intake (or, better, micronutrient concentration in blood) keeps DNA damage to a minimum.