1. Multimedia Data Processing:
1.1 Flow Chart of Multimedia Data Processing
In this research, we derive unstructured video data and corresponding structured e-commerce data from TikTok. Machine learning techniques (computer vision and natural language processing) automatically extract verbal and nonverbal variables. Then, the extracted variables are taken as inputs for interpretable machine learning, XGBoost, and SHAP.
1.2 The Measurement of Beauty
Influencers’ beauty is detected by the beauty score detection API provided by Face++. Utilizing 83 facial landmarks that mark the outline of the face, eye, eyebrow, lip, and nose contour, Face++ outputs beauty scores automatically. Figure 2 displays an example of facial landmarks and beauty score detection interface on Face++.
1.3 The Measurement of Body Motion
The body motion is obtained through the dense optical flow algorithm. By analyzing two adjacent frames of a live streaming video, this algorithm computes a displacement vector for each pixel in the image, generating an optical flow map (see Figure 3). Video 1 demonstrates an example of dense optical flow, with the intensity of the red areas on the right indicating the magnitude of the influencer’s body motion.
2. Interpretable Machine Learning
Interpretable Machine learning methods, eXtreme Gradient Boosting (XGBoost) and SHapley Additive exPlanations (SHAP) were applied to explore the the importance, effect, and interactions of variables.

Figure 4. Density scatter plots of the SHAP values

Figure 5. SHAP Interaction Plot
3. Experimental Stimuli in Study 3
In Study 3, two 2x2 experiments, study3a and study3b, were conducted to test the causality effect. We display the transcript stimuli of benefit appeal, beauty, and Body motion in the following sections.
3.1 Benefit appeals in influencer’s transcript stimuli
To create different extents of using benefit appeal (high-benefit versus low-benefit), we constructed a pair of live streaming transcripts emphasizing benefit appeal (high-benefit) and using fewer benefit appeals (low-benefit) respectively. The content emphasizing benefit appeal in the high-benefit group is marked in blue.
3.2 Beauty of the influencer in picture stimuli
To manipulate the beauty of the influencer, makeup and beautification techniques were applied to the face of the same influencer. The portraits of the influencer in high-beauty condition and low-beauty condition are shown in Figure 6.

Figure 6. The manipulate of influencer's beauty
3.3 Body motion of the influencer in video stimuli
To manipulate the influencer’s body motion, we asked the influencer to behave in different magnitudes of body motion (high-body motion versus low-body motion). We originally recorded four videos in Chinese (high-benefit × low-body motion, high-benefit × high-body motion, low-benefit × low-body motion, and low-benefit × high-body motion). These four videos have been converted into English versions. All eight Chinese and English videos are shown below:
(1) Chinese Version
Group1
Group2
Group3
Group4
(2) English Version