What is 'tokenization' in NLP?
- Breaking text into smaller units
- Compressing files
- Encrypting text
- Translating languages
Explanation
Tokenization
splits text into tokens such as words or subwords for processing.
Related MCQs
- Unsupervised Learning
- Deep Learning
- Supervised Learning
- Reinforcement Learning
اس سوال کو وضاحت کے ساتھ پڑھیں
- Adaptive Interface
- Augmented Information
- Artificial Intelligence
- Automated Intelligence
اس سوال کو وضاحت کے ساتھ پڑھیں
- John McCarthy
- Marvin Minsky
- Geoffrey Hinton
- Alan Turing
اس سوال کو وضاحت کے ساتھ پڑھیں
- Translate languages
- Compute fast
- Exhibit human-like intelligence
- Store data
اس سوال کو وضاحت کے ساتھ پڑھیں
- Artificial General Intelligence
- A spam email filter
- A self-aware robot
- A conscious machine
اس سوال کو وضاحت کے ساتھ پڑھیں
