Bonsoir à tous,

je veux effectuer un fine-tuning de BERT sur mon jeu de données qui est divisé en train.csv et test.csv afin de prédire les traits de personnalité à travers ce jeu de données. Cependant, dans mon code, j'ai rencontré un problème au niveau de la création du modèle.

voici mon code

import pandas as pd
import numpy as np
import tensorflow as tf
from transformers import TFAutoModel, AutoTokenizer

# Load data from CSV files
train_df = pd.read_csv('train.csv', encoding='latin-1')
test_df = pd.read_csv('test.csv', encoding='latin-1')

# Initialize BERT model and tokenizer
model = TFAutoModel.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# Preprocess data
def preprocess_text(text):
    return tokenizer(text, padding=True, truncation=True, return_tensors='tf')

# Apply tokenization to training and test data
train_inputs = preprocess_text(train_df['STATUS'].tolist())
test_inputs = preprocess_text(test_df['STATUS'].tolist())

# Preprocess personality labels if necessary
label_mapping = {'n': 0, 'y': 1}
train_labels = train_df[['cEXT', 'cNEU', 'cAGR', 'cCON', 'cOPN']].replace(label_mapping).values.astype(np.float32)
test_labels = test_df[['cEXT', 'cNEU', 'cAGR', 'cCON', 'cOPN']].replace(label_mapping).values.astype(np.float32)

# Create TensorFlow datasets
train_dataset = tf.data.Dataset.from_tensor_slices(({
    'input_ids': train_inputs['input_ids'],
    'token_type_ids': train_inputs['token_type_ids'],
    'attention_mask': train_inputs['attention_mask']
}, train_labels))
test_dataset = tf.data.Dataset.from_tensor_slices(({
    'input_ids': test_inputs['input_ids'],
    'token_type_ids': test_inputs['token_type_ids'],
    'attention_mask': test_inputs['attention_mask']
}, test_labels))

# Shuffle and batch the datasets
BATCH_SIZE = 64
train_dataset = train_dataset.shuffle(len(train_df)).batch(BATCH_SIZE)
test_dataset = test_dataset.batch(BATCH_SIZE)

class BERTForClassification(tf.keras.Model):

    def __init__(self, bert_model, num_classes):
        super().__init__()
        self.bert = bert_model
        self.dropout = tf.keras.layers.Dropout(0.1)
        self.fc = tf.keras.layers.Dense(num_classes, activation='softmax')

    def call(self, inputs, training=False):
        outputs = self.bert(inputs, training=training)[0]  # Use all hidden states
        pooled_output = outputs[:, 0, :]  # Use the [CLS] token representation
        pooled_output = self.dropout(pooled_output, training=training)
        return self.fc(pooled_output)

# Initialize the classification model
num_classes = len(train_df[['cEXT', 'cNEU', 'cAGR', 'cCON', 'cOPN']].columns)
classifier = BERTForClassification(model, num_classes=num_classes)

# Compile the model with a valid loss function
loss = tf.keras.losses.CategoricalCrossentropy()
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-5)
classifier.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])

# Train the model
history = classifier.fit(
    train_dataset,
    epochs=3
)

# Evaluate the model on test data
classifier.evaluate(test_dataset)

problème 

/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertModel: ['cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias']
- This IS expected if you are initializing TFBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBertModel were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertModel for predictions without further training.
Epoch 1/3
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss. If you're using `model.compile()`, did you forget to provide a `loss` argument?

Merci d'avance

Le message d'erreur a le mérite de vous expliquer que  faire pour le corriger. Est ce que cela a été fait?

Malheureusement ,je n'ai pas encore été corrigé. Pouvez-vous m'aider à résoudre cette erreur et merci d'avance

Les indications données dans le message d'erreur me semble assez claires (à moi). Possible que vous ayez des difficultés à comprendre que faire  mais sans plus de précisions, je ne vois pas trop quoi.

Est ce tu peux m'expliquer comment je peux corriger ce Code pour résoudre mon problème ?