To be sure, market forces certainly affect creative output. On the scientific side, agencies and organizations do this by incentivizing creativity through funding opportunities and prizes. On the industry side, the market can promise untold riches.
But how should creative people view market forces? Should they create for the market, or create things that could potentially open up entirely new markets? And if one is interested in creative outputs, how can we assess the size of creative landscapes and how many good ideas are in certain areas? How could we drive creativity to fill critical needs or incentivize blue-sky high risk/high reward ideas?
Social scientists or economists may have answers to some of these questions, but I want to look at these issues in broader terms, with market forces being more than business opportunities. When it comes to creative innovation, foundations, federal agencies, companies of all sizes, universities, and individuals are all players in the market forces space.
It is important to learn how this landscape operates, since you are undoubtedly part of the landscape. And you are either driving the markets or being driven by them.
Breakthroughs and break-withs
Let’s start by asserting that market forces rarely produce break-withs, but market forces can be used to, or at least attempt to, produce breakthroughs. What’s the difference? Breakthroughs represent a development and a new insight, solving an existing question or a problem that looked hard or impossible to solve.
A break-with is a step above a break-through. A break-with is a conceptual advance that breaks with the ideas that were at the very center of the old way of thinking. The old way of thinking becomes useless or outmoded when the new way kicks in. In extreme cases, new domains form and a new order sets in. In physics, for example, quantum mechanics was a break-with.
The goal of curated conferences: Driving agendas fostering crosslinking
The unstated goal of frontier knowledge scientific conferences is to produce breakthroughs or set the agenda for breakthroughs.
Some have been legendary. The 1965 Dartmouth Summer Research Project on Artificial Intelligence gave birth to the term “artificial intelligence.” Some are known for a name representing their location, like the Cold Spring Harbor Symposia on Quantitative Biology (covering genetics, molecular biology, neuroscience, and cancer research) or the Asilomar Conferences on Recombinant DNA.
The goal of these conferences is to survey and assess the set of knowledge of everything that is known in an area at a given time, especially knowledge that pushes against the frontier of what is known. This is reminiscent of the idea of the adjacent possible, a concept stemming from evolutionary biology. The region adjacent to the boundary, the adjacent possible, represents what is possible based on combinations of ideas at the frontier of the set. In biological evolution, this is how species evolve, little by little.
Some organizations have tried to monetize this idea: bring together top scientists and technologists to imagine inventions which could exist but do not yet exist and license them. That was the initial idea behind Intellectual Ventures (before it evolved into a patent troll venture).
There is an appetite and a market to curate events designed to operate at the adjacent possible. Consider TED, Aspen, Dialog, the World Economic Forum (Davos), and Renaissance Weekends. Events like these can foster the emergence of new ideas by bringing together individuals from diverse backgrounds, disciplines, and areas of expertise. Unstructured networking and serendipitous connections may lead to cross-pollination of ideas.
The idea is that frontier ideas will merge with other frontier ideas. Ideas can fit in existing markets or, in rare instances, may open new markets.
Fostering creativity, incentivizing innovations
Incentivizing creative innovations through prizes and awards is an effective strategy that has been employed across various domains. Consider the XPrize for private spaceflight, the Archon Genomics XPrize for genomic sequencing, and the Carbon Removal XPrize for innovative solutions to remove carbon dioxide from the atmosphere. In mathematics, we have Millennium Prize Problems for solving a set seven problems in mathematics that each carry a $1 million prize for their solution. Google has offered several prizes to incentivize innovation, including for landing a rover on the moon, advancing quantum computing, or coming up with technological solutions to improve the lives of people with disabilities.
Funding is a powerful driver. Funding agencies are in the business of incentivizing creative ideas that have most potential impact. The Defense Advanced Research Projects Agency (DARPA), the research and development agency of the US Department of Defense, and the National Science Foundation (NSF), typify different approaches. DARPA’s funding philosophy is geared towards high-risk, transformative research for defense applications; NSF’s philosophy centers on supporting fundamental, curiosity-driven research across various scientific disciplines. DARPA scouts for people and forms teams. NSF is a more democratic bottom-up approach, with the peer-review process a crucial part of the funding decision.
Assessing the sizes of market of ideas
Academic meetings and professional conferences are the ways in which one can assess the scientific market of ideas. The goal is to get a broad up-to-the-minute representation of the ideas in some discipline. But if these conferences represent the market of ideas, their approaches in how the curate and present this market differ wildly. These approaches might be termed as convergent and divergent.
Some conferences encourage open submissions, a divergent approach, while others have a vision of the future, with assigned sessions and chairs, in what might be termed a convergent approach. Many large conferences, such as American Physical Society, use a keyword/topic-based submission system. Authors submit abstracts and select keywords that match the topic/focus of their research.
Others, like the American Chemical Society, organize sessions around specific topics first and then invite submissions that fit those sessions. This allows more control over the program structure, but people must fit in prescribed niches.
The top computer science conferences, like ACM, work more like a journal editorial board that makes synchronized decisions for each batch of papers. In what may be unique aspect to computer science conferences, submissions are blind; there are no author names on papers. Only the program committee chair knows the author’s names.
At the broadest level, we have the annual meeting of the American Association for the Advancement of Science. Here people submit possible sessions and a lineup of speakers, and topics should be of broad interest.
These are examples of how markets are curated. If a creative innovator wants to design or invent something new for the market, they must understand how the market is created.
Balancing approaches to encourage breakthroughs and break-withs
There are pros and cons to all of the above. Everything depends on the intended objective. A balanced approach is likely best, where there is some structure but also openness. Funding agencies and conferences may want to identify key priority areas but leave room for some flexibility and exploration of new ideas. And all the organizers should periodically reevaluate processes to make sure they are not constraining important emerging research directions or missing opportunities due to rigid formats.
The ideal ratio of market-driven vs radical creativity depends on operating space, be it an industry, the R&D component of a large organization, a foundation that supports research, or a federal agency, such as the National Institutes of Health or the Department of Defense. Thus, in some instances, it may make sense that most resources should go towards incremental improvements, with a smaller portion reserved for high-risk/high-reward moonshots. Others, such as the Hertz foundation, may have as a mission to have rigorous vetting and allocate resources to people who may drive transformative projects.
That same balance should be found in creative teams and organizations. There is often value in having some people focused on understanding current markets deeply while others explore blue sky visions for the future. Communication between the two groups helps balance market viability and radical creativity.
Market-driving approaches
Market-driven innovation refers to companies developing new products, services, or business models mainly in response to current market demands, trends, and competition. Some if not most research fit in this category as well. It tends to result in useful but incremental improvements.
The most successful innovations often open entirely new markets rather than just optimizing for existing ones. Breakthrough innovations tend to come from pushing boundaries and thinking differently, not just iterating on existing ideas. The Ford Model T created a mass market for automobiles and launched the automobile industry. The Sony Walkman created the portable audio player market. Airbnb created a new market for short-term home and apartment rentals and disrupted the hotel/lodging industry. The iPhone ignited the smartphone revolution and defined the modern smartphone and all the apps that come with them.
Market landscapes can shift abruptly with major new innovations. So, creators must strike a balance between studying current realities and envisioning alternate futures that redefine markets.
The most successful organizations that resist stagnation tend to incorporate both market awareness and boundary-pushing innovation. At their best, market-driving innovations take current information and address needs people didn’t even know they had, fundamentally disrupting existing markets. They require a pioneering spirit and a willingness to take risks on groundbreaking but unproven ideas.
In some cases, following the market is not an option. Fiction book writing provides an extreme example; it is impossible to read the market. The advice is always “write the book you want to read” because book market forces are notoriously fickle. At the other extreme, luxury markets drive markets rather than being driven by them.
The most important lesson: Learn how the creative-market forces landscape operates. Then find the right balance between creating for the market and creating for markets that do not yet exist. Because in the end, you are either driving or you are being driven.
