Experimenting with a speech interactive NFT project on Kusama

Introduction

Hi, I am Xinyue, the founder of wordtoworld.io, the project is an extension of my bachelor’s and master’s project at Kunsthochschule Berlin Weissensee, and I joined DesignFarm Berlin, a design-in-tech accelerator in Berlin Germany in October 2020.

I at the same time is also a crypto enthusiast, I heard a lot about Polkadot and Kusama, and I like the experiment on Kusama and NFT,
I believe Kusama is a great place to integrate art, game and tech, thus considering experimenting with blockchain technology on Kusama, the project is called “1001”.

As the technology is complex, and the ecosystem is big, I want to get some feedback here and make sure the project is in the right direction.

Background

I studied product design and interned at Google as a hardware interaction designer in Mountain View, California. I love to create new interactive experiences by using emerging technologies, and design products with playfulness, naivety and sincerity.

My Bachelor project “Scribbling Speech” turns real-time speech into animated drawings, it was featured by Google Experiment (AI collection) and was exhibited at Google I/O 2019 video tent.

I continued this topic and did my Master’s project “Word to World”, which visualizes the spoken words into dynamic 3D animations. It opens up a new interactive experience of story-telling, and I decided to found a startup and develop a speech-interactive product for parents and kids to tell stories.

Please click on the image to watch the video.

Currently, we have an UX designer, an animator, a sound designer, a unity developer, and a backend developer in the team.

About Project 1001

In this part, I will introduce the project “1001” facilitated by KSM token.

1001 transforms spoken words into dynamic stories in real-time using speech recognition and natural language processing, users can generate stories with NFT models, the generated story itself is also an NFT / a group of NFTs. A user who has the story could bid to exhibit it on a land to the whole world.

Please click on the image to watch the video.

Features

• Speech interactive: let a bird fly just by saying "The bird is flying!", 1001 visualize a dynamic world according to users' spoken words.
• Composable NFT story: NFTs from different networks could be integrated together into one story, the story itself as grouped NFTs could be used in multiple DeFi cases like lending, staking, or as an IDO permission.
• DIY unique characters: Make your own characters by composing different body parts, see them come to life right away in your story.
• Rentable land for stories: There are various lands for renting and telling stories. Users can bid for exhibiting their story for the land, and the story will be recorded in "world history".

Marketplace

Story Exposure

Land renting

Model Editor

Token Usages

In the game, the KSM token is used in all the cases involved with the economy, specifically:

• A story could be minted as an NFT on different lands with KSM
• The story elements like Action, Emoji, Sounds could be exchanged as NFT on the marketplace.
• A story on different lands outputs different stories, and the land could be bought or rented with KSM, so that a story could be displayed on the land.
• Our model editor enables users to easily create models in 1001 and models could be minted as an NFT with KSM as fee.

A paratheread?

As far as I know, Substrate is a great tool for building the project, which provides great flexibility in utilizing the blockchain technology, but at the same time issuing a token involves lots of work, which may be beyond our teams' capability. So we would like to use the KSM as the system coin to facilitate the whole system.

We do want to benefit from the cross-chain ecosystem and extend the potential of the project, for example, enable importing NFTs from different networks into a story. Considering smart contract's functionality is limited, and a Parachain may need a lot of funds, is Parathread a better solution for us?

Roadmap

We will need to complete the NFT minting and composing logic, the land buying and renting logic in the next 6 months. As the non-crypto part of the game logic is almost completed in wordToWorld, at that time we will be able to have a playable mobile app game. Model Editor and DeFi utility will come in the second phase.

Funding

I suppose the 1001 project could make NFTs more interactive, add exposure, and further enable NFT creation just by speaking by non-tech and non-art users so it could extend the NFT usability on Kusama.

While some of my engineer friends are interested in working on this project, I am planning to get funds for this experiment.

I’m wondering if it is sensible to get funds from the Kusama treasury for our experimenting?

Disclaimer: The wordToWorld project is funded by the accelerator and venture capitals, we will not use the funds here to support wordToWorld development.

Q & A

How do I balance the time between wordToWorld and 1001?

wordToWorld and 1001 share the same language engine, the back-end is very similar, and wordToWorld is already running at a good pace. In addition to that, on the 1001 side, I will have a blockchain technical expert together with me for all the design and architecture.

I wouldn’t merge the two projects in the future because WordToWorld is an educational game for children and 1001 is an NFT project for us crypto users.

Will the code be all open-sourced?

The blockchain part built on/with Substrate will all be open-sourced, but the language engine and the front-end app cost the team a lot of effort, we would rather keep it in private.

If you are interested in working with the project, always welcome to contact me by [email protected]