This research will adopt a combined theoretical analysis and experimental validation design. First, based on the theoretical framework of reinforcement learning and history-dependent tasks, the reasons for low exploration efficiency will be analyzed, and an improved exploration strategy will be proposed. Second, experiments will be conducted using simulated environments and real datasets to validate the performance of the improved strategy in different history-dependent tasks. Third, comparative experiments will be used to evaluate the differences between this strategy and traditional methods in terms of exploration efficiency and task performance. The API will be used to support data preprocessing, model training, and result visualization, enhancing research efficiency and reproducibility. Finally, based on the experimental results, optimization directions and application recommendations for the improved strategy will be proposed.