The Brain-Computer Interface (BCI) is a system which transforms the brain activity of different mental tasks into a control signal. The system provides an augmentative communication method for patients with severe motor disabilities. In this paper, a neural classifier based on improved particle swarm optimization (IPSO) is proposed to classify an electroencephalogram (EEG) of mental tasks for left hand movement imagination, right hand movement imagination, and word generation. First, the EEG patterns utilize principle component analysis (PCA) in order to reduce the feature dimensions. Then a three-layer neural network trained using particle swarm optimization is used to realize a classifier. The proposed IPSO method consists of the modified evolutionary direction operator (MEDO) and the traditional PSO. The proposed MEDO combines the evolutionary direction operator (EDO) and the migration. The MEDO can strengthen the searching global solution. The IPSO algorithm can prevent premature convergence and outperform the other existing methods. Experimental results have shown that our method performs well for the classification of mental tasks from EEG data.