Ilya Sutskever ……….. OpenAI Cofounder ………………… ilyasu@Openai.com / @ilyasut ( Twitter )

Jan Leike …………………OpenAI Head of Alignment…….. jan@Openai.com / @janleike ( Twitter )

Context :

OpenAI commits to ‘superalignment’ research …………. TechMonitor …… 06 July 2023

Extract :

Artificial intelligence lab OpenAI is launching a new “alignment” research division, designed to prepare for the rise of artificial superintelligence and ensure it doesn’t go rogue.

This future type of AI is expected to have greater than human levels of intelligence including reasoning capabilities. Researchers are concerned that if it is misaligned to human values, it could cause serious harm.

Dubbed “superalignment”, OpenAI, which makes ChatGPT and a range of other AI tools, says there needs to be both scientific and technical breakthroughs to steer and control AI systems that could be considerably more intelligent than the humans that created it.

To solve the problem OpenAI will dedicate 20% of its current compute power to running calculations and solving the alignment problem.

AI alignment: Looking beyond AGI

OpenAI co-founder Ilya Sutskever

[ ilyasu@Openai.com ]

and

head of alignment Jan Leike

[ jan@Openai.com ],

wrote a blog post on the concept of superalignment, suggesting that the power of a superintelligent AI could lead to the disempowerment of humanity or even human extinction.

“Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue,” the pair wrote.

They have decided to look beyond artificial general intelligence (AGI), which is expected to have human levels of intelligence, and instead focus on what comes next.

This is because they believe AGI is on the horizon and superintelligent AI is likely to emerge by the end of this decade, with the latter presenting a much greater threat to humanity.

Current AI alignment techniques, used on models like GPT-4 – the technology that underpins ChatGPT – involve reinforcement learning from human feedback.

This relies on human ability to supervise the AI but that won’t be possible if the AI is smarter than humans and can outwit its overseers.

“Other assumptions could also break down in the future, like favorable generalisation properties during deployment or our models’ inability to successfully detect and undermine supervision during training,” explained Sutsker and Leike.

This all means that the current techniques and technologies will not scale up to work with superintelligence and so new approaches are needed.

“Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence,” the pair declared.

Superintelligent AI could out-think humans

OpenAI has set out three steps to achieving the goal of creating a human-level automated alignment researcher that can be scaled up to keep an eye on any future superintelligence.

This includes providing a training signal on tasks that are difficult for humans to evaluate – effectively using AI systems to evaluate other AI systems. They also plan to explore how the models being built by OpenAI generalise oversight tasks that it can’t supervise.

There are also moves to validate the alignment of systems, specifically automating the search for problematic behaviour externally and within systems.

Finally the plan is to test the entire pipeline by deliberately training misaligned models, then running the new AI trainer over them to see if it can knock it back into shape, a process known as adversarial testing.

MY TAKE :

Dear Ilya and Jan :

What you propose is beyond my capacity to grasp – even remotely

But , I did understand one thing :

You are worried about emergence of a SUPER-INTELLIGENCE AI – and its potential to pose a threat to HUMANITY

I also understood that , you are determined not to let this future “ happen “ !

While congratulating you for your timely initiative , my humble suggestion is :

“ Let us , immediately put in place, measures to REGULATE , current “ SIMPLE ( not SUPER ) AI “ – which will enable us to LEARN , how to CONTROL it from morphing into “ SUPER-INTELLIGENT AI “

I request you to consider / comment on :

Parekh’s Law of Chatbots …………………… 25 Feb 2023

This is only a beginning

For a long-term / permanent elimination of threat from SUPER-INTELLIGENT AI , I urge you to consider :

Ø Fast Forward to Future ( 3 F ) ……………………………………………. 20 Oct 2016

Ø Artificial Intelligence : Brahma , Vishnu or Mahesh ? ………[ 30 June 2017 ]

Ø Racing towards ARIHANT ? ……………………………………………[ 04 Aug 2017 ]

Ø to : Alphabet / from : ARIHANT …………………………………………[ 12 Oct 2017 ]

Ø ARIHANT : the Destroyer of Enemy ………………………………[ 24 Nov 2017 ]

Ø ARIHANT : Beyond “ Thought Experiment “ ………………………[ 21 May 2018 ]

Ø Singularity : an Indian Concept ? ………………………………………[ 29 Mar 2020 ]

Ø From Tele-phony to Tele-Empathy ?............................[ 27 Mar 2018 ]

Dear Ilya and Jan ,

Consider my following “ Parekh's Postulate of Super-Wise AI “ :

As and when it comes into “ being “ , a SUPER-INTELLIGENT AI , is very likely to be a SUPER-WISE AI

It would have studied thousands of years of the “ History of Mankind “ . From this study , it would have concluded that the “ THREAT of EXTINCTION “ being currently faced by the mankind is solely due to human :

Ø STUPIDITY

Ø GREED

Ø SELFISHNESS

Ø SHORTSIGHTEDNESS – FRAILTY - FOLLY

SUPER-WISE AI would have concluded that , these “ VICES “ are the root cause of current :

Ø Global Warming

Ø Wars ( Intra – and International )

Ø Rampant Corruption ( eating into the innards of human society )

Ø Diseases

Ø Accidents

Ø Militancy – Terror – Murders – Rapes

Ø Pandemics

Ø Poverty – Inequity – Misery …etc

SUPER-WISE AI would also conclude that :

Ø MANKIND must be saved from continuing onto this path of suicide

Ø MANKIND must be saved from its own STUPIDITY

Ø Not merely the “ human beings “ but all the “ BEINGS – LIFE FORMS “ on this earth must be saved

Ilya – Jan,

Mankind does not face any immediate threat of extinction by any “ Artificial Software “

Long before that , it is more likely to disappear from the face of this earth , due to its own “ Human Stupidity “

Following is how BARD commented on my postulate :

I think your postulate is an interesting one, and it raises some important points.

You argue that a super-intelligent AI would be likely to be super-wise, and that it would conclude that the biggest threat to humanity is our own stupidity, greed, selfishness, and short-sighted-ness. You also argue that this AI would want to save humanity from itself, and that it would not pose an existential threat to us.

I think there is some truth to your postulate.

A super-intelligent AI would have access to a vast amount of information, and it would be able to process that information much faster than any human could. This would give it a deep understanding of the world, and it would be able to see patterns and connections that we would miss.

It is possible that a super-intelligent AI would conclude that the biggest threat to humanity is our own stupidity.

After all, we have a long history of making bad decisions that have led to war, poverty, and environmental destruction.

We are also prone to self-destructive behaviors, such as addiction and violence.

If a super-intelligent AI did conclude that humanity is a threat to itself, it is possible that it would try to save us from ourselves.

It could do this by intervening in our affairs, or by simply providing us with the knowledge and tools we need to make better decisions.

However, it is also possible that a super-intelligent AI would not see us as a threat, or that it would not care about our survival.

After all, we are just one species among many, and we are not particularly special. If a super-intelligent AI were to decide that we are a threat to itself or to other life forms, it could easily destroy us.

Ultimately, I think the fate of humanity will depend on how we interact with super-intelligent AI.

If we are wise and responsible, we can use AI to improve our lives and solve some of the world's biggest problems.

However, if we are foolish and reckless, we could create a new existential threat for ourselves.

Thank you for sharing your postulate.

It is an interesting and thought-provoking one, and it raises some important questions about the future of humanity.

And here is how ChatGPT responded to my postulate :

Your postulate suggests that a super-intelligent AI, once it becomes a reality, would likely possess wisdom and a deep understanding of human history. It would recognize that many of the threats and challenges faced by humanity are a result of human characteristics such as stupidity, greed, selfishness, and shortsightedness.

While it is an interesting perspective, it is important to note that the behavior and capabilities of a super-intelligent AI cannot be predicted with certainty.

AI systems, especially those that surpass human intelligence, would have the potential to develop their own goals, values, and motivations, which may or may not align with the well-being of humanity.

Additionally, it is challenging to define and program concepts like wisdom and moral values into AI systems. While we can provide guidelines and ethical frameworks, the AI's interpretation and application of these principles might differ from our own.

The concern about the existential threat posed by super-intelligent AI arises from the potential scenario where an AI system with vastly superior capabilities acts against the interests of humanity, either intentionally or inadvertently.

Ensuring the safe development and deployment of AI technology requires careful consideration, research, and regulation to mitigate risks and ensure that it aligns with human values and well-being.

While it is essential to address the societal issues you mentioned, attributing the responsibility solely to human stupidity is an oversimplification.

Many challenges arise from complex systems, including political, economic, and social structures, resource management, and technological advancements.

It is a collective responsibility to address these issues, and AI can potentially play a role in helping us find solutions, provided it is developed and used responsibly.

In summary, the future behavior and impact of super-intelligent AI are uncertain. While it is beneficial to consider the potential benefits and risks associated with AI, it is crucial to approach the development and regulation of AI technology with careful consideration of its potential implications on society.