Yerzhan Rzagaliyev, MSc student Thesis Supervisor: Prof. Huseyin Atakan Varol Department of Robotics Nazarbayev University Neural Network based robust control for dual axis reaction wheel inverted pendulum Outline 2 • Motivation • Physical Setup • System Dynamics • System Identification • Neural Network Implementations • Simulations • Experimental Results Motivation 3  Industry 4.0 Internet of Things (IoT) Big Data Cloud computing Augmented and virtual reality (AR/VR) Additive manufacturing (AM) Human-Robot interaction (HRI) Motivation Variable Impedance/Stiffness Actuators  For safe HRI used new robots like variable impedance actuators (VIA) (Lasota et al, 2017)  Highly nonlinear  Decrease of positioning accuracy  Vibrations Reaction wheel integration  It has benefits to implement reaction wheel-integrated VSA (Baimyshev et al, 2016)  Can boost the performance of the VSA 4 Reaction wheel inverted pendulum Motivation Figure 1. Cubli balancing on a corner (Muehlebach, D’Andrea, 2016) Figure 3. Self balancing dual axis pendulum (Turkmen et. Al, 2017) Figure 2. Self balancing single axis inverted pendulum (Belascuen, Aguilar, 2018) 5 Design of the setup Lower Part  3D printed  8 springs  Connected with universal joint  2 encoders Figure 4. Lower part of the system 6 Design of the setup Upper part  3D printed  Steel flywheel  Diameter – 8 cm  Mass – 160 g  2 encoders  2 Maxon motors  2 Vex gyroscopes  20 pieces  Mass – 50 g  Diameter – 15 mm  Thickness – 9 mm Figure 5. Upper part of the system Figure 6. Mass holders 7 Figure 7. Experimental setup (Baimukashev et al, 2020) Design of the setup 8 System Dynamics Kinematics Lagrangian function 9 System Dynamics In order to control stabilization of the pendulum, we need to define all affecting forces and torques, and we have equation: By deriving dynamics from Lagrangian function, we have our state-space representation as: 10 System Identification In Solidworks Mass – 1.49 kg Center of mass (in z axis) – 31 cm Moments of inertia:  Ixx = 0.19 kg·m2  Iyy = 0.20 kg·m2  Izz = 0.01 kg·m2 Figure 8. Mechanical properties 11 System Identification Figure 9. Defining rotational spring constant and pendulum friction coefficient. Real world result (blue dashed line) vs simulated result (red line) In MATLAB Spring constant  k1 – 12.5 N·m/rad  k2 – 6.9 N·m/rad Pendulum friction coefficient  b1 – 0.09 N·m·s/rad  b2 – 0.12 N·m·s/rad 12 Parameter Symbol Value Unit Mass of the whole pendulum m p 1.49 kg Center of mass along z-axis l p 0.31 m Rotational spring constant around x-axis k 1 12.5 N ·m/rad Rotational spring constant around y-axis k 2 6.9 N ·m/rad Friction coefficient on x-axis of the pendulum b 1 0.09 N ·m·s/rad Friction coefficient on y-axis of the pendulum b 2 0.12 N ·m·s/rad Friction coefficient of the reaction wheel b ω1 1.11E-04 N ·m·s/rad Friction coefficient of the reaction wheel b ω2 9.40E-05 N ·m·s/rad Motor torque constant K 0.13 N ·m/A Moment of inertia of the pendulum I p 0.19 kg·m2 Moments of inertia of the reaction wheels I 1 = I 2 7.53E-05 kg·m2 System Identification Table 1. Nominal parameters of the system 13 Neural Network Implementations Nominal NN (FNN)  3 layers fully connected  128 neurons  ReLU  RMSprop – 0.0008  4800 nominal parameters trajectories  DGX-2 100 epochs Robust NN (RNN)  2 layers fully connected + 2 layers Long Short-Term Memory (LSTM)  128 neurons  ReLU  RMSprop – 0.0008  2000 nominal + 7000 changed trajectories  DGX-2 100 epochs Table 2. The comparison of FNN and RNN for the nominal NN for 100 trajectories (failure rate) 14 Optimal control problem (OCP) trajectories  MATMPC (Chen et al, 2019)  8000 trajectories, after filtering 5300 clean  15000 trajectories, after filtering 8000 clean  Mass – 0 to 30 %, spring constants – -20 to 0 %, friction coefficients – -20 to 20 %  For testing 500 trajectories for each (nominal and changed) Nominal Changed FNN based nominal NN 2 56 2 68 RNN based nominal NN 3 56 13 70 Simulations 15 Init_data 500 traj Nominal NN Nominal parameters 478 (95.6 %) Changed parameters 233 (46.6 %) Robust NN Nominal parameters 482 (96.4 %) Changed parameters 420 (84 %) Different initial conditions for simulating the NNs 500 trajectories with six random states:  Angular positions – θ1 and θ2  Angular velocities – θ’1 and θ’2  Wheel velocities – ω1 and ω2 Table 3. The success rate Nominal NN and Robust NN for 500 random trajectories Nominal parameters, nominal NN Changed parameters, nominal NN Nominal parameters, robust NN Changed parameters, robust NN Figure 10. Simulation results (red dot – fail, blue dot – success) Simulations 16 Simulations Figure 11. Histogram of costs of nominal NN and robust NN Nominal parameters:  Nominal NN (478)  Mean – 1.16  Std – 0.16  Robust NN (482)  Mean – 1.28  Std – 0.20  Changed parameters:  Nominal NN (233)  Mean – 1.18  Std – 0.20  Robust NN (420)  Mean – 1.29  Std – 0.21 17 Simulations Figure 12. 3D plot of parameter simulation. (dark blue – fail, yellow – success)  Average of system parameters on x and y axes  Mass – from 0 to 30 %  Spring constants – from -20 to 0 %  Pendulum friction coefficients – from -20 to 20 % 18 Mass (grams) 0 100 200 300 Nominal NN (x1e-4) 21.65 26.27 28.64 29.46 Robust NN (x1e-4) 21.99 22.83 26.31 27.32 Mass (grams) 0 100 200 300 Nominal NN 10 6 4 0 Robust NN 10 9 5 0 Experiments Table 5. Average cost of Nominal NN and Robust NN for 10 different trajectories. Table 4. The success rates of Nominal NN and Robust NN for 10 different trajectories 19 Future work and Conclusion  Conduct experiments with:  Putting balanced mass  Changed springs  Changed pendulum friction coefficient  Reaction wheel integration improves VSA/VIA robots positioning accuracy  Robust NN can deal with parameter uncertainties and has more accuracy comparing with nominal NN (~40 % more) 20 Thank you! 21 References 22 Lasota, P. A., Fong, T., & Shah, J. A. (2017). A survey of methods for safe human-robot interaction. Now Publishers. Baimyshev, A., Zhakatayev, A., & Varol, H. A. (2016). Augmenting variable stiffness actuation using reaction wheels. IEEE Access, 4, 4618-4628. Muehlebach, M., & D’Andrea, R. (2016). Nonlinear analysis and control of a reaction-wheel-based 3-D inverted pendulum. IEEE Transactions on Control Systems Technology, 25(1), 235-246. Belascuen, G., & Aguilar, N. (2018, June). Design, modeling and control of a reaction wheel balanced inverted pendulum. In 2018 IEEE Biennial Congress of Argentina (ARGENCON) (pp. 1-9). IEEE. Türkmen, A., Korkut, M. Y., Erdem, M., Gönül, Ö., & Sezer, V. (2017, November). Design, implementation and control of dual axis self balancing inverted pendulum using reaction wheels. In 2017 10th International Conference on Electrical and Electronics Engineering (ELECO) (pp. 717-721). IEEE. References 23 Baimukashev, D., Sandibay, N., Rakhim, B., Varol, H. A., & Rubagotti, M. (2020, July). Deep Learning-Based Approximate Optimal Control of a Reaction-Wheel-Actuated Spherical Inverted Pendulum. In 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (pp. 1322-1328). IEEE. Yutao Chen, Mattia Bruschetta, Enrico Picotti, and Alessandro Beghi. Matmpc- a matlab based toolbox for real-time nonlinear model predictive control. In 2019 18th European Control Conference (ECC), pages 3365–3370. IEEE, 2019. Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Slide 6 Slide 7 Slide 8 Slide 9 Slide 10 Slide 11 Slide 12 Slide 13 Slide 14 Slide 15 Slide 16 Slide 17 Slide 18 Slide 19 Slide 20 Slide 21 Slide 22 Slide 23