Code Walkthrough of Text Prompting Subnet

In this section we present a step-by-step code walkthrough of the Text Prompting Subnet.

Explorer

You can see the text prompting subnet in action on the Taostats explorer (select Subnet 01: Text Generation from top right menu).

Before you proceed

If you are new to Bittensor subnets and building blocks, read the following sections before you proceed further:

• Bittensor Building Blocks.
• Anatomy of Incentive Mechanism.

For ease of understanding, we will focus on how a single subnet validator interacts with multiple subnet miners in this subnet.

Subnet summary

The text prompting subnet works like this:

• The subnet validator sends text prompts to subnet miners and waits for the responses from the subnet miners.
• After receiving the responses from the subnet miners, the subnet validator scores and ranks the responses from the subnet miners.
• Finally, the subnet validator sends these ranks to the blockchain, where the Yuma Consensus allocates the rewards to the participating subnet miners and subnet validators.

Use of large language models

Both the subnet validator and the subnet miners use large language models to create the prompt text strings (subnet validator) and respond to the prompts with prompt completions (subnet miners).

Start

After you install Bittensor, create a wallet, register to be a subnet validator and ensure that you have enough stake to be a subnet validator, you will execute a command like below to start validating in the text prompting subnet. For the exact command, see Bittensor Validator Setup Guide,

python3 /path/to/validator.py <options>

The above command runs the following code segment of the validator.py:

def main():
    neuron().run()

The above code executes the following steps:

• First, initializes a neuron object.
• Then starts the neuron.run() method. The run() calls the run_forward(), which then calls the forward() function.

Before we look at the initialization part of the code, let's first summarize the run phase where we can see the key actions centered on a subnet validator in this incentive mechanism.

Subnet validator run summary

The forward() function performs the following steps (see the below diagram):

1. Prompt for summary

• The subnet validator sends a body of text to a set of subnet miners with a prompt asking the subnet miners to summarize these sentences.
• The subnet validator scores the responses containing summaries from the subnet miners.
• The subnet validator then updates the weights of the subnet miners. These weights are not yet sent to the blockchain. This establishes the context for this entire run.

2. Prompt for question

• The subnet validator takes the best scoring summary text, selects a separate set of subnet miners and sends them a prompt asking them to craft a question based on the summary text.
• The subnet validator then scores the question responses and locally updates the weights of these subnet miners.

3. Prompt for answer

• Next, the subnet validator takes the best scoring question from the above step.
• The subnet validator then selects a separate set of subnet miners, non-overlapping with any that were already prompted in the above steps, and prompts these subnet miners for answers to the question.
• The subnet validator then scores the answer responses and locally updates the weights of these subnet miners.

4. Repeat

• Repeat steps 2 and 3 multiple times.

This completes a single run, i.e., a single iteration of the forward() method. At the conclusion of the single run of forward(), the subnet validator sets weights on chain.

tip

Hence, in a single run, i.e., in a single iteration of the forward() method, the subnet validator prompts multiple times, each time selecting a separate set of subnet miners, prompting them, scoring their responses and updating their weights locally.

See also Subnet validator run details.

Initialization

The following steps are executed in the intialization of the neuron object. Refer to the code that initializes a neuron object.

Create a subnet validator instance

• A neuron object is created, using the configuration passed in the command line arguments list for the above validator.py. This is managed by the bt.Config method. Setting up the configuration includes the target hardware, getting the information of the validator name, wallet, the subtensor network (mainchain or testchain name) and so on.
• When the neuron object is created, it means a subnet validator node instance is created with the subnet validator identity contained in the hotkey and the netuid of the subnet.

Use the specified hardware device

This below device initialization section makes sure that the subnet validator instance runs on the target hardware device specified:

# Init device.
    bt.logging.debug("loading", "device")
    self.device = torch.device(self.config.neuron.device)
    bt.logging.debug(str(self.device))

Connect with the blockchain and initialize

This below subtensor initialization section starts a connection from this subnet validator to the blockchain.

self.subtensor = bt.subtensor(config=self.config)

Establish the identity

This below section makes sure that you, who is running this code, have a Bittensor wallet and it is registered. Otherwise, you are not allowed to participate on the mainchain.

self.wallet = bt.wallet(config=self.config)

Initialize the metagraph

Next, we sync with the metagraph of the subnet. The metagraph is a neural network graph object. It contains comprehensive information about all the participants in the subnet. We sync with this subnet's metagraph because we must know all the subnet miners that are in this text prompting subnet. We pass the netuid to the bt.metagraph method, identifying which subnet we belong to, and download the subnet's metagraph into our local subtensor (i.e., local blockchain).

In this below initialize metagraph section the Bittensor method bt.metagraph is called on the current netuid.

self.metagraph = bt.metagraph( 
    netuid=self.config.netuid, network=self.subtensor.network, sync=False
    )  # Make sure not to sync without passing subtensor
    self.metagraph.sync(subtensor=self.subtensor)  # Sync metagraph with subtensor.

Initialize weights

Next, because you are a subnet validator, you will maintain a vector of weights for all the subnet miners. Each element of this vector is a weight, a floating point number, for a subnet miner. At the end of each step of the validation run (see Subnet validator summary) you will update this vector as a moving average. So, first start by setting these moving average weights to zero.

self.moving_averaged_scores = torch.zeros((self.metagraph.n)).to(self.device)

Establish the context for prompting

We know from Prompt for summary that the very first thing a subnet validator does is to establish a common context with the subnet miners ("What's the topic we will be talking about?").

The subnet validator extracts some random lines of text from Dataset.

self.dataset = Dataset()

OpenWebText and RedPajama-Data

The Dataset class downloads datasets from OpenWebText and RedPajama-Data.

Gating model

The gating model is intended to be used in inference, when a subnet validator wants to pre-determine a set of subnet miners that are suitable for the inference task at hand. The GatingModel is a neural network model that is trained to identify such subnet miners.

Create axon server

Next, the subnet validator activates an axon server on itself. This is the first step in estabilishing the network connectivity between itself and all the subnet miners in this subnet.

In this block of code, replicated below:

if not self.config.neuron.axon_off:
    bt.logging.debug("serving ip to chain...")
    try:
        axon = bt.axon(wallet=self.wallet, config=self.config)
        try:
            self.subtensor.serve_axon(
                netuid=self.config.netuid,
                axon=axon,
            )
        except Exception as e:
            bt.logging.error(f"Failed to serve Axon with exception: {e}")
            pass
        del axon
    except Exception as e:
        bt.logging.error(
            f"Failed to create Axon initialize with exception: {e}"
        )
        pass
else:
    bt.logging.debug("axon off, not serving ip to chain.")

When the subnet validator calls axon = bt.axon(wallet=self.wallet, config=self.config) (line 4 above) this means that an API server is spawned on this subnet validator node with the name axon. The axon server will now begin to serve on behalf of this subnet validator, i.e., this subnet validator is ready to receive queries from any client worldwide.

Next, when this subnet validator executes the subtensor command subtensor.serve_axon(netuid=self.config.netuid, axon=axon) (line 6 above), this subnet validator is:

• Passing its axon information to the blockchain (subtensor) it is connected to (see Connect with the blockchain and initialize).
• Telling (subtensor.serve_axon()) its subtensor to "turn on this axon server on my behalf, open the IP:port information contained in this axon so the outside world can send requests to me."

Create dendrite client

The axon acts mainly as a server. However, during the process of prompting the subnet miners and receiving responses from them, this subnet validator needs a way to query these subnet miners. This is accomplished by the subnet validator by means of instantiating dendrite client on itself.

self.dendrite = bt.dendrite(wallet=self.wallet)

Initialize the reward model

Next, the reward model is initialized. The reward model determines the subnet incentive mechanism. The reward functions used in the reward model determine how a response from a subnet miner must be processed to generate the reward for this subnet miner. See the below diagram that shows how a response from a subnet miner is processed to compute the reward.

Journey from response to reward

The below diagram shows the journey of a subnet miner's response to the reward for a single prompt run. As described in Subnet validator run summary, a full single run of forward() consists of 9 such prompt runs. Hence, only after 9 such response-to-reward journeys are completed will a single forward() run be complete.

A few key notes on this reward model section of the code:

• Reward model consists of three types of functions that process the response from the subnet miner:
  • Reward functions: The outputs of these functions are floating point numbers. The output of one function is added to the output of another function. A reward weight typically does not entirely zero out the corresponding reward function, only adjusts the contribution of the reward function.
  • Masking functions: The outputs of these functions are either 0 or 1. These are non-negotiable functions. These outputs are multiplied with the response. Hence, if a masking function outputs a 0, then it will zero out the response from the subnet miner. These masking functions reflect whether the subnet miner is blacklisted, or caught with a NSFW (not safe for work) element in the response, or similar such safeguards.
  • Penalty functions: The outputs of these functions are floating point numbers. These outputs are multiplied with the response.
• The subnet validator must decide which reward functions to use and add that function name in the list. See below code section:

  self.reward_weights = torch.tensor(
      [
          self.config.reward.dpo_weight,
          self.config.reward.rlhf_weight,
          self.config.reward.reciprocate_weight,
          self.config.reward.dahoas_weight,
          self.config.reward.prompt_based_weight,
      ],
      dtype=torch.float32,
    ).to(self.device)
• Important: Set the reward function weight for each reward function you have added to the list. Make sure that however many reward functions you added to the list, their reward_weights must all sum to one.
• Similarly, for penalty functions, see:

  self.penalty_functions = [
      TaskValidationPenaltyModel(max_penalty=0.1),
      ContentMatchPenaltyModel(max_penalty=0.1),
      KeywordMatchPenaltyModel(max_penalty=1),
  ]

With this the initialization code section concludes.

Subnet validator run details

Before you proceed

See Subnet validator run summary before you proceed.

Recall that when the subnet validator starts off, the validator.py Python code executes this below code:

def main():
    neuron().run()

In the above code, first the neuron() function creates a neuron object, i.e., a subnet validator node instance, where the initialization happens.

Next, the neuron().run() method is run, which is when the subnet validator begins the validation process.

In the run mode, the following steps are executed:

• A self check is performed to ensure that the subnet validator is registered and possesss proper hotkeys.
• The run_forward() section calls the forward function.
• This forward function will then:
  • Start with subnet miner weights all initialized to zero.
  • Perform 9 distinct prompt runs as described in the Subnet validator run summary.
  • Then it will set the subnet miner weights on chain. See should_set_weights.

Common actions in all prompt runs

In all the multiple prompt runs that occur within a single run of the forward(), the following applies:

• Each prompt is sent to a distinct, non-overlapping, set of subnet miners. Hence, a given subnet miner is not prompted more than once within a given forward() run. (Not used) Gating model
• At the end of each prompt run the subnet validator computes the rewards for all the subnet miners that participated in this prompt run. The subnet validator maintains these rewards as a reward vector.
• The reward vectors between any two prompt runs within a given forward() run are updated on a moving average basis. See this code section:

  # Update moving_averaged_scores with rewards produced by this step.
    # shape: [ metagraph.n ]
    alpha: float = self.config.neuron.moving_average_alpha
    self.moving_averaged_scores: torch.FloatTensor = alpha * scattered_rewards + (
        1 - alpha
    ) * self.moving_averaged_scores.to(self.device)
• Within a forward() run, any subsequent prompt run usually extracts the best completion from the immediate previous prompt run, within the same forward() run, and makes use of this best completion.

Select miners

Recall that in the initialization section the subnet validator has downloaded the metagraph of the subnet. See Initialize the metagraph.

This metagraph contains the uids of all the active subnet miners in the subnet. A random list of uids is extracted from this metagraph. See the below code section:

  # Get the list of uids to query for this step.
   uids = get_random_uids(self, k=k, exclude=exclude).to(self.device)
   axons = [self.metagraph.axons[uid] for uid in uids]

Gather the axons of the subnet miners

To prompt a subnet miner, the subnet validator must know the axon information of the subnet miner to send prompt query to. The above line axons = [self.metagraph.axons[uid] for uid in uids] pulls together the axon server information of all the above-selected uids into a list object axons.

Create synapse object

Next, a Synapse object synapse is created for constructing the prompt.

synapse = prompting.protocol.Prompting(roles=["user"], messages=[prompt])

The above Prompting class is a subclass of the Synapse class. See the code section in protocol.py that begins with the following line:

class Prompting(bt.Synapse):

This Synapse object synapse will serve as the vehicle for exchanging information between the subnet validator and subnet miners. When the subnet miners receive this synapse object, they will update the completion field of the object with their responses (prompt completions).

Send the prompt

Now it is time to send out the prompt to the selected subnet miners. Recall from the initialization section Create dendrite client that the subnet validator created the dendrite client instance on itself. In the below code section the subnet validator broadcasts the synapse object to the axons of the selected subnet miners:

# Make calls to the network with the prompt.
responses: List[bt.Synapse] = await self.dendrite(
    axons=axons,
    synapse=synapse,
    timeout=timeout,
)

The subnet validator waits until the timeout has elapsed before processing the responses list.

Score the responses

The text prompt completions from the subnet miners are first cleaned up. See the code section that begins with the below line:

for response in responses:
    # remove leading and trailing periods
    completion = response.completion.strip(".")

Next, the responses are processed using the reward model described in Initialize the reward model.

Send subsequent prompts

As described in Subnet validator run summary, a series of 9 prompts are sent, and the subnet miner responses are scored according to the reward model.

Common actions in all prompt runs

Also see Common actions in all prompt runs.

Send updated weights to subtensor

When the 9 prompt runs are completed within this single forward() run, the code returns back to the validator.py. See below:

# Run multiple forwards.
async def run_forward():
    coroutines = [
        forward(self)
        for _ in range(self.config.neuron.num_concurrent_forwards)
    ]
    await asyncio.gather(*coroutines)
    self.loop.run_until_complete(run_forward())
    # Resync the network state
    if should_checkpoint(self):
        checkpoint(self)
    # Set the weights on chain.
    if should_set_weights(self):
        set_weights(self)
        save_state(self)
    # Rollover wandb to a new run.
    if should_reinit_wandb(self):
        reinit_wandb(self)

The set_weights() function on line 14 above calls the self.subtensor.set_weights() function that transmits the subnet miner weights to the subtensor (blockchain). See the below code section in the weights.py:

# Set the weights on chain via our subtensor connection.
self.subtensor.set_weights(
    wallet=self.wallet,
    netuid=self.config.netuid,
    uids=processed_weight_uids,
    weights=processed_weights,
    wait_for_finalization=False,
    version_key=validators.__spec_version__,
)